Unmanned aerial vehicle candidate action safety gate method and system based on world model prediction

Abstract

The application relates to the technical field of unmanned aerial vehicle autonomous flight control and intelligent decision-making, and provides an unmanned aerial vehicle candidate action safety gating method and system based on world model prediction. The method comprises the following steps: acquiring a plurality of groups of candidate control action sequences generated by a world model, and corresponding predicted flight trajectory data of each group of candidate actions; performing constraint checking on each group of candidate action sequences from a preset dimension; screening the candidate control action sequences that pass the constraint checking as executable action outputs; if all candidate action sequences do not pass the safety gating checking, triggering a preset classical flight controller to take over the unmanned aerial vehicle flight control and outputting a basic stable control instruction. The method performs systematic gating checking on the candidate actions from four dimensions of physical limits, environmental safety, task accuracy and actuator state, comprehensively covers various constraint boundaries of flight safety, and effectively reduces safety risks such as attitude over-limit, collision out-of-bound and actuator saturation caused by the world model output actions.

Description

Technical Field

[0001] This application relates to the field of autonomous flight control and intelligent decision-making technology for unmanned aerial vehicles (UAVs), and in particular to a safety gating method and system for candidate UAV actions based on world model prediction. Background Technology

[0002] In the field of autonomous flight control for unmanned aerial vehicles (UAVs) based on world models, existing solutions primarily focus on improving the prediction accuracy and candidate action generation efficiency of the world model, lacking a systematic safety verification mechanism for the output candidate actions. Some safety control solutions only set single-dimensional constraint checks, failing to simultaneously cover multiple constraints such as the physical limits of the UAV, environmental obstacle risks, mission accuracy requirements, and actuator working boundaries, making it difficult to comprehensively avoid flight risks caused by prediction biases in the world model. Furthermore, when all candidate actions generated by the world model fail to meet safety requirements, existing solutions lack a reliable smooth degradation and takeover mechanism, which can easily lead to loss of control output and threaten flight safety. Summary of the Invention

[0003] This application provides a safety gating method and system for candidate actions of unmanned aerial vehicles (UAVs) based on world model prediction. It aims to solve the problem that in the field of autonomous flight control of UAVs based on world models, existing solutions mostly focus on improving the prediction accuracy and candidate action generation efficiency of the world model, but lack a systematic safety verification mechanism for the output candidate actions.

[0004] In a first aspect, embodiments of this application provide a safety gating method for candidate actions of a UAV based on world model prediction, applied to an autonomous flight control system of a UAV equipped with a world model prediction module, the method comprising: Obtain multiple sets of candidate control action sequences generated by the world model, as well as the predicted flight trajectory data corresponding to each set of candidate actions; Each candidate action sequence is constrained and verified from preset dimensions. The constraint verification includes physical constraint verification to check whether the attitude angle, linear velocity and acceleration exceed the preset physical limits of the UAV; environmental constraint verification to check whether the predicted trajectory intrudes into the preset obstacle safety distance and no-fly zone; task constraint verification to check whether the trajectory meets the preset waypoint tracking and hovering accuracy requirements; and actuator constraint verification to check whether the control quantity causes actuator saturation or sudden change in control quantity. The preset dimensions include physical constraints, environmental constraints, task constraints and actuator constraints. Candidate control action sequences that pass the constraint verification are selected as executable action outputs; if all candidate action sequences fail the safety gating verification, the preset classic flight controller is triggered to take over the UAV flight control and output basic stability control commands.

[0005] In some embodiments, obtaining multiple sets of candidate control action sequences generated by the world model, and the predicted flight trajectory data corresponding to each set of candidate actions, includes: generating multiple sets of candidate control action sequences within the dynamic imagination budget limit of the current control cycle, each set of sequences containing control commands for multiple consecutive control cycles; inputting each set of candidate control action sequences into the world model to perform forward inference, and obtaining the predicted flight trajectory, attitude change sequence and risk assessment data corresponding to each set of actions.

[0006] In some embodiments, the physical constraint verification of attitude angle, linear velocity, and acceleration to see if they exceed preset UAV physical limits includes: extracting the predicted attitude angle, linear velocity, and acceleration values ​​corresponding to the candidate action sequence step by step; comparing each parameter with a preset UAV physical limit threshold one by one; if any parameter at any time exceeds the corresponding limit threshold, it is determined that the candidate action sequence has failed the physical constraint verification.

[0007] In some embodiments, the environmental constraint verification of whether the predicted trajectory intrudes into the preset obstacle safety distance and no-fly zone includes: constructing the spatial boundary between the obstacle-occupied area and the no-fly zone based on real-time environmental perception data; calculating the minimum distance between the predicted flight trajectory and each obstacle boundary at each sampling point, and determining whether the minimum distance is less than a preset safety distance threshold; determining whether the predicted trajectory enters the no-fly zone; if the trajectory intrudes into the safety distance or enters the no-fly zone, determining that the group of candidate action sequences has failed the environmental constraint verification.

[0008] In some embodiments, the process of verifying whether the mission constraint verification trajectory meets the preset waypoint tracking and hovering accuracy requirements includes: extracting the target waypoint sequence or hovering position coordinates in the current mission objective; calculating the tracking error between the predicted flight trajectory and each target waypoint, as well as the position deviation range during the hovering phase; if the tracking error or position deviation exceeds the preset accuracy threshold, determining that the corresponding candidate action sequence has failed the mission constraint verification.

[0009] In some embodiments, the actuator constraint verification of whether the control quantity causes actuator saturation or control quantity mutation includes: converting each control quantity in the candidate control action sequence into the output command of the corresponding actuator; checking whether the output command of each actuator exceeds its maximum output range to determine whether actuator saturation occurs; calculating the change amplitude of the control quantity between adjacent control cycles to determine whether it exceeds a preset mutation threshold; if actuator saturation or control quantity mutation exceeds the limit, determining that the corresponding candidate action sequence has failed the actuator constraint verification.

[0010] In some embodiments, the step of selecting candidate control action sequences that pass constraint verification as executable action output includes: after completing constraint verification for all candidate action sequences, collecting all candidate action sequences that pass all verifications; if there are multiple sets of sequences that pass verification, selecting the set with the highest comprehensive score as executable action output, wherein the comprehensive score is calculated based on trajectory tracking accuracy, energy consumption level and safety margin.

[0011] In some embodiments, after the candidate control action sequence that has passed the constraint verification is selected as the executable action output, the method further includes: continuously collecting feedback data on the actual flight state of the UAV during the execution of the executable action; comparing the actual state with the world model prediction state at the corresponding time and calculating the real-time execution deviation; if the execution deviation is within the allowable range, maintaining the current world model control mode; if the deviation continues to expand, dynamically shrinking the imagined budget scale of subsequent control cycles.

[0012] In some embodiments, the step of triggering a preset classic flight controller to take over the UAV flight control and output basic stability control commands if all candidate action sequences fail the safety gating verification includes: generating a mode switching trigger signal when all candidate action sequences fail the safety gating verification; gradually reducing the output weight of the world model control quantity and simultaneously increasing the output weight of the classic flight controller control quantity; after a preset transition period, completely switching to the classic flight controller to output control commands, and having the classic controller execute basic stability flight actions such as hovering, deceleration, or return to home.

[0013] Secondly, this application provides a safety gating system for candidate drone actions based on world model prediction, the system comprising: The sequence acquisition unit is used to acquire multiple sets of candidate control action sequences generated by the world model, as well as the predicted flight trajectory data corresponding to each set of candidate actions; The constraint verification unit is used to perform constraint verification on each group of candidate action sequences from preset dimensions. The constraint verification includes physical constraint verification to check whether the attitude angle, linear velocity and acceleration exceed the preset physical limits of the UAV; environmental constraint verification to check whether the predicted trajectory intrudes into the preset obstacle safety distance and no-fly zone; task constraint verification to check whether the trajectory meets the preset waypoint tracking and hovering accuracy requirements; and actuator constraint verification to check whether the control quantity causes actuator saturation or control quantity abrupt change. The preset dimensions include physical constraints, environmental constraints, task constraints and actuator constraints. The instruction output unit is used to filter candidate control action sequences that pass the constraint verification as executable action outputs; if all candidate action sequences fail the safety gating verification, the preset classic flight controller is triggered to take over the UAV flight control and output basic stability control instructions.

[0014] The method provided in this application has the following beneficial effects: First, multi-dimensional and comprehensive constraint verification. Candidate actions are systematically gating and checking from four dimensions: physical limits, environmental safety, mission accuracy, and actuator status. This comprehensively covers all kinds of constraint boundaries for flight safety, effectively reducing safety risks such as attitude overshooting, collision overshooting, and actuator saturation caused by actions output by the world model.

[0015] Second, the safety degradation mechanism is reliable. When all candidate actions fail to meet safety requirements, the classic flight controller is automatically triggered to smoothly take over, avoiding single-point failure of the learning control strategy and ensuring that the UAV always has basic stable flight capabilities.

[0016] Third, the verification logic is clear and interpretable. It adopts a layered gating process of "action generation - multi-dimensional verification - optimal output - failure takeover", and each verification rule is clear and traceable, which facilitates engineering implementation and security verification.

[0017] Fourth, it has strong versatility and adaptability. The gating mechanism does not depend on a specific world model structure and can be combined with various world model prediction architectures, adapting to different types of UAV flight control platforms.

[0018] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Attached Figure Description

[0019] To more clearly illustrate the technical solutions of the embodiments of this application, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a schematic flowchart illustrating the steps of a safety gating method for candidate drone actions based on world model prediction, provided in one embodiment of this application. Figure 2 This is a schematic diagram illustrating the principle of a safety gating method for candidate drone actions based on world model prediction, provided in one embodiment of this application. Figure 3 This is a schematic block diagram of a safety gating system for candidate drone actions based on world model prediction, provided in one embodiment of this application. Figure 4 This is a schematic block diagram of the structure of an unmanned aerial vehicle (UAV) autonomous flight control system provided in one embodiment of this application.

[0021] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and do not limit this application. Detailed Implementation

[0022] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0023] The flowchart shown in the attached diagram is for illustrative purposes only and does not necessarily include all content and operations / steps, nor does it necessarily have to be performed in the order described. For example, some operations / steps can be broken down, combined, or partially merged, so the actual execution order may change depending on the actual situation.

[0024] It should be understood that, in order to clearly describe the technical solutions of the embodiments of the present invention, the terms "first" and "second" are used in the embodiments of the present invention to distinguish identical or similar items with essentially the same function and effect. Those skilled in the art will understand that the terms "first" and "second" do not limit the quantity or execution order, and the terms "first" and "second" are not necessarily different.

[0025] It should be understood that the terminology used in this specification is for the purpose of describing particular embodiments only and is not intended to limit the scope of the application. As used in this specification and the appended claims, the singular forms “a,” “an,” and “the” are intended to include the plural forms unless the context clearly indicates otherwise.

[0026] It should also be understood that the term “and / or” as used in this application specification and the appended claims means any combination of one or more of the associated listed items and all possible combinations, and includes such combinations.

[0027] In the field of autonomous flight control for unmanned aerial vehicles (UAVs) based on world models, existing solutions primarily focus on improving the prediction accuracy and candidate action generation efficiency of the world model, lacking a systematic safety verification mechanism for the output candidate actions. Some safety control solutions only set single-dimensional constraint checks, failing to simultaneously cover multiple constraints such as the physical limits of the UAV, environmental obstacle risks, mission accuracy requirements, and actuator working boundaries, making it difficult to comprehensively avoid flight risks caused by prediction biases in the world model. Furthermore, when all candidate actions generated by the world model fail to meet safety requirements, existing solutions lack a reliable smooth degradation and takeover mechanism, which can easily lead to loss of control output and threaten flight safety.

[0028] Please refer to Figure 1The UAV candidate action safety gating method based on world model prediction provided in this embodiment is applied to an autonomous flight control system of a UAV equipped with a world model prediction module. This system can be deployed on an onboard computing unit or partially deployed on a ground control station and execute commands through the onboard flight control unit. Figure 2 As shown, the system includes at least a world model prediction module, a candidate action sequence generation unit, a security gating verification module, an executable action output unit, a classic flight controller, and a UAV actuator; the modules interact with each other via an airborne bus, and the control cycle can be configured from 20ms to 100ms according to the UAV platform's computing power and mission requirements.

[0029] The provided method for safe gating of candidate drone actions based on world model prediction includes steps S101 to S103. Details are as follows: Step S101. Obtain multiple sets of candidate control action sequences generated by the world model, as well as the predicted flight trajectory data corresponding to each set of candidate actions.

[0030] Specifically, this step is the input preparation phase for safety gating. Its core is to obtain multiple sets of candidate actions and their corresponding future state trajectories derived from the world model within the available computing resources and prediction confidence range of the current control cycle. The specific implementation process is as follows: Input data preprocessing: At the start of each control cycle, the current flight status data, environmental observation data, and mission target data of the UAV are first collected. Flight status data includes at least one of the following: UAV's 3D position, 3D velocity, attitude quaternion, three-axis angular velocity, three-axis acceleration, motor speed, battery voltage and remaining power, and flight control operating mode. Environmental observation data includes at least one of the following: visual images, depth point clouds, lidar ranging data, ultrasonic ranging data, wind speed and direction estimates, obstacle detection results, local occupancy maps, and no-fly zone geofence data. Mission target data includes at least one of the following: target waypoint sequence, target tracking pose, hovering coordinates, inspection area boundary, return-to-home coordinates, and safe landing point coordinates. The above data is timestamped, transformed to the aircraft coordinate system or the Northeast-Northeast geographic coordinate system, and filtered for noise reduction before being used as input for subsequent world model prediction and candidate action generation.

[0031] Candidate action sequence generation: Within the limit of the dynamic imagination budget corresponding to the current control cycle, multiple groups of candidate control action sequences are output through a candidate action generation strategy. The dynamic imagination budget is used to limit the number of candidate action groups, the prediction time domain length, the action search space range and the optimization iteration times allowed to be generated in the current cycle. The candidate control action sequence contains control instructions for multiple consecutive control cycles, which specifically may include at least one of the desired total thrust, three-axis control moment, desired attitude angle, desired velocity, and waypoint correction. The candidate action can be generated in multiple ways such as random sampling, cross-entropy method optimization, policy network sampling, gradient descent optimization, etc. When the imagination budget is high, more groups of candidate action sequences with longer time domains can be generated; when the imagination budget is low, only a small number of groups of candidate action sequences with short time domains are generated.

[0032] World model forward deduction: Each generated group of candidate control action sequences is sequentially input into the UAV world model to perform forward deduction calculation, so as to obtain the predicted flight trajectory data corresponding to each group of candidate actions. The world model is a pre-trained dynamics prediction model, which can adopt one or a combination of a recurrent state space model, a Transformer world model, a physics-constrained hybrid world model, and a probabilistic dynamics model. It can output a predicted state sequence of H control steps in the future based on the current UAV state and the input control sequence, specifically including data such as predicted position, velocity, attitude angle, angular velocity, acceleration, risk assessment value, and collision probability at each moment. Each group of candidate actions corresponds to an independent predicted trajectory, and all predicted trajectories are stored in association with the corresponding candidate actions for subsequent safety gating verification.

[0033] Step S102. Perform constraint verification on each group of candidate action sequences from preset dimensions; the constraint verification includes physical constraint verification for checking whether the attitude angle, linear velocity and acceleration exceed the preset physical limits of the UAV, environmental constraint verification for checking whether the predicted trajectory invades the preset obstacle safety distance and no-fly zone, task constraint verification for checking whether the trajectory meets the preset waypoint tracking and hovering accuracy requirements, and actuator constraint verification for checking whether the control quantity causes actuator saturation or control quantity mutation; the preset dimensions include physical constraints, environmental constraints, task constraints and actuator constraints.

[0034] Specifically, this step is the core verification phase of the safety gating, which adopts a multi-dimensional parallel verification mechanism to sequentially verify each group of candidate action sequences and their corresponding predicted trajectories from the four preset dimensions of physical constraints, environmental constraints, task constraints, and actuator constraints. The constraint verifications of the four dimensions are in a logical AND relationship, and only when all dimensions pass the verification, the group of candidate action sequences is judged to be qualified. The specific verification rules for each dimension are as follows: Physical constraint verification: This is used to verify whether the predicted flight state corresponding to the candidate action exceeds the inherent physical limits of the UAV, avoiding risks such as attitude instability and structural overload. In practice, state parameters such as predicted attitude angle, linear velocity, angular velocity, linear acceleration, and angular acceleration corresponding to the candidate action sequence are extracted step-by-step. Each parameter is compared with a preset physical limit threshold for the UAV. These physical limit thresholds are pre-calibrated based on the mechanical structure and power system parameters of the UAV platform. For example, they can be set to: maximum roll angle ±35°, maximum pitch angle ±30°, maximum horizontal flight speed 18 m / s, maximum vertical ascent speed 5 m / s, maximum vertical descent speed 3 m / s, maximum linear acceleration 8 m / s², and maximum angular velocity 120° / s. If the absolute value of any parameter at any time exceeds the corresponding limit threshold, the candidate action sequence is directly determined to have failed the physical constraint verification, terminating the verification process for subsequent dimensions.

[0035] Environmental constraint verification: This checks whether the predicted trajectory corresponding to the candidate action intrudes into dangerous areas of the environment to avoid collisions and airspace violations. In practice, firstly, based on real-time environmental perception data and pre-stored airspace data, the spatial boundaries of obstacle-occupied areas and no-fly zones are constructed. Obstacles can be divided into static and dynamic obstacles. Static obstacle boundaries can be expanded outwards by a preset safety distance through inflation processing. Dynamic obstacles require simultaneous prediction of their future H-step motion trajectory and construction of dynamic boundaries. No-fly zones include controlled airspace, height-restricted areas, and electronic fence areas, stored as polygonal boundaries or grid maps. Then, the minimum Euclidean distance between the predicted flight trajectory and each obstacle boundary is calculated point by point, and it is determined whether the minimum distance is less than a preset safety distance threshold. Simultaneously, it is determined whether the coordinates of the predicted trajectory fall within the no-fly zone. If the trajectory intrudes within the obstacle safety distance or enters the no-fly zone, the candidate action sequence is deemed to have failed the environmental constraint verification.

[0036] Task constraint verification: This checks whether the predicted trajectory corresponding to the candidate action meets the accuracy and rule requirements of the current task, preventing task execution failure. In practice, it extracts constraints such as the target waypoint sequence, hovering position coordinates, tracked target pose, and inspection path from the current task objective. It calculates the tracking error between the predicted flight trajectory and each target waypoint, including horizontal position error, altitude error, arrival time error, as well as the position deviation range during hovering and the target tracking viewpoint deviation range. These errors are compared with preset task accuracy thresholds. For example, for waypoint tracking tasks, the waypoint arrival radius error can be set to no more than 3m; for hovering tasks, the horizontal position deviation can be set to no more than 1m and the altitude deviation to no more than 0.5m; and for target tracking tasks, the camera viewpoint deviation can be set to no more than ±20°. If the tracking error or position deviation exceeds the preset accuracy threshold, the candidate action sequence is determined to have failed the task constraint verification.

[0037] Actuator constraint verification: This is used to verify whether candidate control actions exceed the physical output capacity of the actuator, or whether sudden changes in control quantities cause actuator shocks and control oscillations. In practice, each control quantity in the candidate control action sequence is first converted into the corresponding actuator output command using a power allocation matrix. For multi-rotor UAVs, this involves distributing the total thrust and three-axis torque into the speed commands of each motor; for fixed-wing UAVs, it involves converting them into commands such as control surface deflection and throttle opening. Then, it checks whether the output commands of each actuator exceed its maximum and minimum output range to determine if actuator saturation has occurred. Simultaneously, it calculates the change in control quantities between adjacent control cycles to obtain the rate of change and determines whether it exceeds a preset threshold for sudden changes. For example, the thrust change rate can be set to no more than 20% of the maximum thrust per control cycle, and the torque change rate to no more than 15% of the maximum torque per control cycle. If actuator saturation or excessive control quantity sudden changes occur, the candidate action sequence is deemed to have failed the actuator constraint verification.

[0038] Step S103. Select candidate control action sequences that pass the constraint verification as executable action outputs; if all candidate action sequences fail the safety gating verification, trigger the preset classic flight controller to take over the UAV flight control and output basic stability control commands.

[0039] Specifically, this step is the output decision stage of safety gating. Based on the verification results, the final output control command is selected, and a degrade takeover mechanism is set up to ensure flight safety. The specific implementation process is as follows: Qualified Candidate Action Screening and Ranking: After completing four rounds of constraint verification for all candidate action sequences, all candidate action sequences that pass all verifications are collected. If no sequence passes verification, the classic flight controller takeover process is directly triggered; if only one set of sequences passes verification, it is directly identified as an executable action; if multiple sets of sequences pass verification, all qualified sequences are ranked using a comprehensive scoring mechanism, and the set with the highest comprehensive score is selected as the executable action output. The comprehensive score is calculated based on a weighted average of three indicators: trajectory tracking accuracy, energy consumption level, and safety margin. The trajectory tracking accuracy score is negatively correlated with tracking error, the energy consumption score is negatively correlated with the predicted total energy consumption, and the safety margin score is positively correlated with the minimum distance between the predicted trajectory and the obstacle and the margin from the physical limit. The weights of each indicator can be dynamically adjusted according to the task type.

[0040] Normal output process: After determining the executable actions, the control command corresponding to the current control cycle in the candidate action sequence is output to the subsequent control fusion module, either directly or after weighted fusion with classical control variables, and then sent to the actuator. Simultaneously, the complete predicted trajectory and scoring results of the candidate actions are stored in a cache for prediction reliability calculation and deviation comparison in the next control cycle.

[0041] Degradation Takeover Trigger Procedure: When all candidate action sequences fail the safety gating check, a mode switching trigger signal is generated, initiating the classic flight controller takeover process. The classic flight controller includes at least one of a PID controller, LQR controller, MPC controller, geometry controller, trajectory tracking controller, and hover controller. It is an engineering-proven stable controller capable of outputting basic stable control commands to ensure the basic flight safety of the UAV. The takeover process employs a smooth transition mechanism, gradually reducing the output weight of the world model control variables while simultaneously increasing the output weight of the classic flight controller control variables, avoiding sudden changes in control variables caused by hard switching. After complete takeover, the classic controller executes basic safe flight actions such as hovering, deceleration, return to home, or safe landing.

[0042] In some embodiments, obtaining multiple sets of candidate control action sequences generated by the world model, and the predicted flight trajectory data corresponding to each set of candidate actions, includes: generating multiple sets of candidate control action sequences within the dynamic imagination budget limit of the current control cycle, each set of sequences containing control commands for multiple consecutive control cycles; inputting each set of candidate control action sequences into the world model to perform forward inference, and obtaining the predicted flight trajectory, attitude change sequence and risk assessment data corresponding to each set of actions.

[0043] This embodiment provides a detailed explanation of the generation process of candidate actions and predicted trajectories, with the implementation scenario being a routine waypoint inspection task for a quadcopter UAV.

[0044] At the start of the current control cycle, the system first calculates the dynamic imagination budget value for the current cycle based on the prediction reliability, historical prediction error, safety margin, and task risk level of the previous cycle. This imagination budget value is normalized to the range of 0-1, corresponding to three specific parameters: the number of candidate action groups, the prediction time domain length, and the number of optimization iterations. When the budget value is the maximum of 1, 16 candidate action sequences are allowed to be generated, the prediction time domain length is 20 control steps, and the number of action optimization iterations is 8. As the budget value decreases, all three parameters are reduced proportionally. When the budget value is below the minimum threshold of 0.2, only 2 candidate action groups are allowed to be generated, the prediction time domain is shortened to 3 steps, and no iterative optimization is performed.

[0045] Starting from the current flight state, the candidate action generation unit generates an initial sequence of candidate actions within a budgeted timeframe and using a policy network sampling method with Gaussian perturbations. Each sequence contains thrust and torque commands corresponding to the time domain length. The 16 candidate actions are sequentially input into the world model for forward inference. The world model, based on a cyclic state space structure, sequentially infers the UAV state for each control step, ultimately outputting a 20-step predicted trajectory for each action, including position, velocity, attitude, and collision risk value at each moment. After all candidate action sequences are bound to their corresponding predicted trajectories, they are input into the safety gating module for verification.

[0046] This embodiment limits the generation scale of candidate actions by using dynamic imagination budget, so that the prediction computation is matched with the credibility of the world model, avoiding wasting computing power on long-term invalid extrapolation in low-credibility scenarios.

[0047] In some embodiments, the physical constraint verification of attitude angle, linear velocity, and acceleration to see if they exceed preset UAV physical limits includes: extracting the predicted attitude angle, linear velocity, and acceleration values ​​corresponding to the candidate action sequence step by step; comparing each parameter with a preset UAV physical limit threshold one by one; if any parameter at any time exceeds the corresponding limit threshold, it is determined that the candidate action sequence has failed the physical constraint verification.

[0048] This embodiment describes the specific execution process of physical constraint verification, with the verification objects being the candidate actions and predicted trajectories generated in Embodiment 1.

[0049] The system pre-stores a table of the UAV's physical limit parameters, including: maximum roll angle ±35°, maximum pitch angle ±30°, maximum yaw rate 180° / s, maximum horizontal velocity 16m / s, maximum vertical climb rate 4m / s, maximum vertical descent rate 2.5m / s, and maximum linear acceleration 6m / s². During verification, for a single set of candidate actions corresponding to a 20-step predicted trajectory, the system sequentially iterates through the state data of each time step from step 1 to step 20. Extract the roll and pitch angle values ​​at that moment, take the absolute value and compare it with the maximum angle threshold; Extract the three-axis velocities at that moment, calculate the resultant velocity and the horizontal resultant velocity, and compare them with the maximum velocity threshold respectively; Extract the triaxial angular velocity and linear acceleration at that moment, and compare them with the corresponding thresholds.

[0050] During the traversal, if any parameter at any time step exceeds the threshold, the candidate action for that group is immediately marked as failing the physical constraint check, the verification of the remaining time steps for that group is terminated, and the verification process for the next group of candidate actions begins directly. If all parameters in all 20 steps do not exceed the threshold, the candidate action for that group is determined to have passed the physical constraint check and proceeds to the next dimension of environmental constraint check.

[0051] This embodiment employs a time-step traversal verification and early termination mechanism, which can reduce invalid calculations and improve verification efficiency while ensuring the comprehensiveness of verification.

[0052] In some embodiments, the environmental constraint verification of whether the predicted trajectory intrudes into the preset obstacle safety distance and no-fly zone includes: constructing the spatial boundary between the obstacle-occupied area and the no-fly zone based on real-time environmental perception data; calculating the minimum distance between the predicted flight trajectory and each obstacle boundary at each sampling point, and determining whether the minimum distance is less than a preset safety distance threshold; determining whether the predicted trajectory enters the no-fly zone; if the trajectory intrudes into the safety distance or enters the no-fly zone, determining that the group of candidate action sequences has failed the environmental constraint verification.

[0053] This embodiment describes the specific execution process of environmental constraint verification, and the implementation scenario is an urban low-altitude inspection scenario that includes static buildings and temporary no-fly zones.

[0054] In each control cycle, the system first extracts the outlines of static obstacles within the current field of view based on the point cloud data from the airborne LiDAR. It then expands the boundary of each obstacle outwards by a safe distance of 1.5 meters to generate an expanded obstacle boundary. Simultaneously, it loads pre-stored polygonal geofence data for urban no-fly zones and height thresholds for temporary height-restricted areas to construct a complete environmental constraint spatial boundary. For detected dynamic obstacles (such as flying birds or other drones), the system predicts their trajectory for the next 20 steps using Kalman filtering and synchronously performs expansion processing on the obstacle position at each step to generate a dynamic constraint boundary.

[0055] During verification, for the predicted trajectory of a single set of candidate actions, the predicted coordinates of the UAV at that moment are extracted step by step, and two judgments are performed sequentially: Calculate the minimum Euclidean distance between the coordinate and the expansion boundaries of all static and dynamic obstacles. If the minimum distance is less than 0, the trajectory is determined to have intruded into the obstacle area. The ray casting method is used to determine whether the coordinates fall within the polygonal area of ​​the no-fly zone, and at the same time, whether the predicted altitude exceeds the height limit threshold.

[0056] If any time step meets any of the above conditions, the candidate action group is determined to have failed the environmental constraint verification; if all time steps meet the safety distance and airspace requirements, the environmental constraint verification is determined to have passed and the task constraint verification stage is entered.

[0057] This embodiment covers three types of environmental risks: static obstacles, dynamic obstacles, and airspace control, and can comprehensively avoid environmental collisions and violation risks of the predicted trajectory.

[0058] In some embodiments, the process of verifying whether the mission constraint verification trajectory meets the preset waypoint tracking and hovering accuracy requirements includes: extracting the target waypoint sequence or hovering position coordinates in the current mission objective; calculating the tracking error between the predicted flight trajectory and each target waypoint, as well as the position deviation range during the hovering phase; if the tracking error or position deviation exceeds the preset accuracy threshold, determining that the corresponding candidate action sequence has failed the mission constraint verification.

[0059] This embodiment describes the specific execution process of task constraint verification, with the implementation scenario being a waypoint tracking task during power line inspection.

[0060] The current mission objective comprises 12 sequentially arranged waypoints. Each waypoint includes the target's latitude and longitude, altitude, and expected arrival time. The mission requires that the horizontal tracking error of the waypoints not exceed 2.5m, the altitude error not exceed 0.8m, and the arrival time deviation not exceed 0.5s. During verification, the predicted position corresponding to the expected arrival time of each target waypoint is first extracted from the predicted trajectory, and the horizontal position deviation, altitude deviation, and time deviation for each waypoint are calculated.

[0061] For flight segments between waypoints, the lateral deviation between the predicted trajectory and the preset inspection path is calculated. For waypoints requiring hovering for photography, the predicted positions of 10 control steps during the hovering phase are extracted, and the standard deviation and maximum deviation of the positions are calculated. All deviation values ​​are compared one by one with the preset task accuracy threshold. If the tracking error of any waypoint exceeds the threshold, or the position deviation during the hovering phase exceeds the allowable range, the candidate action is deemed to have failed the task constraint verification. If all waypoints and flight segments meet the accuracy requirements, the task constraint verification is deemed to have passed, and the process proceeds to the actuator constraint verification stage.

[0062] This embodiment designs verification rules for the multi-dimensional task requirements of inspection tasks, which can ensure that the selected candidate actions meet the task execution quality requirements while ensuring safety.

[0063] In some embodiments, the actuator constraint verification of whether the control quantity causes actuator saturation or control quantity mutation includes: converting each control quantity in the candidate control action sequence into the output command of the corresponding actuator; checking whether the output command of each actuator exceeds its maximum output range to determine whether actuator saturation occurs; calculating the change amplitude of the control quantity between adjacent control cycles to determine whether it exceeds a preset mutation threshold; if actuator saturation or control quantity mutation exceeds the limit, determining that the corresponding candidate action sequence has failed the actuator constraint verification.

[0064] This embodiment describes the specific execution process of actuator constraint verification, and the implementation object is the power execution system of a quadcopter UAV.

[0065] The system pre-stores the power system allocation matrix and actuator parameters: the maximum speed of the four motors is 8000 rpm, the minimum speed is 2000 rpm, and the thrust is proportional to the square of the speed. The power allocation matrix converts the total thrust T and roll, pitch, and yaw moments into the square values ​​of the four motor speeds. During verification, two checks are performed for each control step of a single candidate action sequence: Saturation check: The total thrust and torque command of this step is converted into the target speed of four motors through the allocation matrix and compared with the maximum and minimum speed thresholds respectively. If the target speed of any motor exceeds the range, the actuator is determined to be saturated. Sudden change check: Calculate the difference between the control quantity of this step and the control quantity of the previous control step to obtain the thrust change and the torque change of the three axes. Divide each of these by the maximum value of the corresponding control quantity to obtain the rate of change. If the rate of change of any control quantity exceeds 20% / control cycle, it is determined that the control quantity has exceeded the limit due to a sudden change.

[0066] If actuator saturation or control variable abruptly exceeds the limit occurs at any time step, the candidate action group is deemed to have failed the actuator constraint verification; if all time steps meet the actuator output range and rate of change requirements, the candidate action group is deemed to have passed the constraint verification of all four dimensions and is marked as a qualified candidate action.

[0067] This embodiment verifies the actuator constraints from both output capability and rate of change, which can prevent actuator overload damage and suppress machine oscillation caused by sudden changes in control quantity.

[0068] In some embodiments, the step of selecting candidate control action sequences that pass constraint verification as executable action output includes: after completing constraint verification for all candidate action sequences, collecting all candidate action sequences that pass all verifications; if there are multiple sets of sequences that pass verification, selecting the set with the highest comprehensive score as executable action output, wherein the comprehensive score is calculated based on trajectory tracking accuracy, energy consumption level and safety margin.

[0069] This embodiment describes the filtering and output process for multiple sets of candidate actions, and is applicable to scenarios where there are multiple sets of candidate actions that pass all constraint checks.

[0070] After the security gating system completes four rounds of verification for all candidate actions, it obtains a total of 5 valid candidate action sequences. The system calculates three scoring indicators for each sequence and performs normalization processing: Track tracking accuracy score: Calculate the average waypoint tracking error of the set of trajectories. The smaller the error, the higher the score. Linear normalization is used to map to the interval of 0 to 100 points. This item accounts for 40% of the weight. Energy consumption level score: The total energy consumption is estimated by calculating the total thrust integral value corresponding to this set of trajectories. The lower the energy consumption, the higher the score. The score is normalized to the range of 0 to 100 points. This item accounts for 30% of the total score. Safety margin score: Extract the minimum distance between the trajectory and the obstacle and the minimum margin between the trajectory and the physical limit. The larger the distance and margin, the higher the score. Normalize to the range of 0 to 100 points. This item accounts for 30% of the weight.

[0071] The three scores are weighted and summed to obtain a comprehensive score for each group of candidate actions. The five sequences are sorted from highest to lowest comprehensive score, and the candidate action ranked first is selected as the final executable action. The control instruction corresponding to its current control cycle is output. At the same time, the second and third ranked candidate actions are stored in a cache for hot start optimization in the next cycle.

[0072] This embodiment adopts a multi-index weighted scoring mechanism, which can further select the control action with the best overall performance on the basis of safety and compliance, taking into account task quality, energy efficiency and safety redundancy.

[0073] In some embodiments, after the candidate control action sequence that has passed the constraint verification is selected as the executable action output, the method further includes: continuously collecting feedback data on the actual flight state of the UAV during the execution of the executable action; comparing the actual state with the world model prediction state at the corresponding time and calculating the real-time execution deviation; if the execution deviation is within the allowable range, maintaining the current world model control mode; if the deviation continues to expand, dynamically shrinking the imagined budget scale of subsequent control cycles.

[0074] This embodiment describes the feedback optimization process after the output of executable actions, realizing a closed-loop iteration of control effect and predictive capability.

[0075] During the execution of executable actions, the system continuously collects real flight status feedback data of the UAV through onboard sensors on a control cycle basis, including parameters such as real position, velocity, attitude, and angular velocity. The real state at each moment is timestamped and aligned with the corresponding world model predicted state to unify the coordinate system, and the real-time execution deviation is calculated, including position deviation, velocity deviation, and attitude deviation.

[0076] The system is configured with three levels of deviation handling logic: If the execution deviation is within the allowable range (e.g., horizontal position deviation less than 0.8m, velocity deviation less than 0.5m / s), then the current world model control mode will be maintained, and the imagined budget for the next cycle will remain at the original level. If the execution deviation exceeds the allowable range but is still within the safety threshold, and shows a continuous increasing trend for three consecutive control cycles, it is determined that the prediction accuracy of the world model has decreased. The imagined budget size of subsequent control cycles is dynamically reduced, with each reduction being 20% ​​of the current budget. At the same time, the number of candidate action groups and the prediction time domain length are reduced. If the execution deviation exceeds the safety threshold, the classic controller takeover process will be triggered directly.

[0077] When the subsequent execution deviation gradually decreases and remains within the allowable range for several consecutive cycles, the system gradually increases the imagined budget to restore the predictive ability of the world model.

[0078] This embodiment dynamically adjusts the imagined budget through real-world deviation closed-loop feedback, ensuring that the intensity of the world model's use matches the actual prediction accuracy and preventing the deviation from continuously expanding and causing safety risks.

[0079] In some embodiments, the step of triggering a preset classic flight controller to take over the UAV flight control and output basic stability control commands if all candidate action sequences fail the safety gating verification includes: generating a mode switching trigger signal when all candidate action sequences fail the safety gating verification; gradually reducing the output weight of the world model control quantity and simultaneously increasing the output weight of the classic flight controller control quantity; after a preset transition period, completely switching to the classic flight controller to output control commands, and having the classic controller execute basic stability flight actions such as hovering, deceleration, or return to home.

[0080] This embodiment describes the downgrade takeover process when all candidate actions fail the verification, ensuring the smoothness of the mode switching process.

[0081] Once the safety gating verification is complete, if all 16 candidate action sequences fail at least one constraint verification, a mode switching trigger signal is immediately generated to initiate the smooth takeover process. The system is set to a transition duration of 0.8 seconds. During the transition phase, a linear weight switch is performed: the weight α of the world model control variable gradually decreases from its current value at a fixed slope, while the weight of the classical flight controller control variable synchronously increases from 1 to α. The weight value is updated every 10 ms, and finally, after 0.8 seconds, α drops to 0, completely switching to the classical flight controller to output control commands.

[0082] During the transition phase, the classic flight controller defaults to using the position loop PID controller, outputting hover control commands to maintain the drone's current altitude and gradually decelerate to a hovering state. If the drone is currently at the edge of a no-fly zone or near obstacles, the obstacle avoidance and return-to-home controller is activated to control the drone to decelerate and move towards a safe area. After complete takeover, the system continuously monitors the prediction reliability of the world model and the verification results of candidate actions, and gradually switches back to the world model control mode once the recovery conditions are met.

[0083] This embodiment employs a smooth takeover mechanism with linear weighted transition to avoid sudden changes in control quantities and attitude shocks caused by hard switching, thereby improving flight stability during the degradation process.

[0084] In some embodiments, such as Figure 2 As shown, each control cycle is executed sequentially according to the following process: Status acquisition: Acquire the UAV's current flight status, environmental observation data, mission objectives, and historical control parameters through the onboard sensor array, and complete time alignment and preprocessing; World model prediction: The preprocessed input data is fed into the UAV world model, which then extrapolates the flight status distribution and risk indicators within a limited time domain based on historical window data. Prediction credibility calculation: Combine six indicators, namely model output uncertainty, historical prediction error, out-of-distribution distance, safe envelope distance, execution bias, and sensor health status, to calculate the prediction credibility of the current period and normalize it to the range of 0 to 1. Dynamic imagination budget calculation: Based on prediction confidence, safety margin, task risk, historical prediction error and model uncertainty, calculate the dynamic imagination budget for the current period, with the limit between the minimum budget and the maximum budget; Budget and credibility threshold judgment: Determine whether the current prediction credibility is higher than the preset credibility threshold and whether the dynamic imagination budget is higher than the preset budget threshold; if not, directly jump to the classic controller takeover branch; if satisfied, enter the candidate action generation stage. Candidate action generation: Within the constraints of the dynamic imagination budget, multiple sets of candidate control action sequences are generated, and the corresponding predicted trajectory for each set is obtained through forward extrapolation of the world model; Safety gating verification: Verify each group of candidate actions from four dimensions: physical constraints, environmental constraints, task constraints, and actuator constraints, and select qualified sequences; Verification result judgment: If there are candidate actions that pass all verifications, the group with the highest comprehensive score is selected as the world model control variable; if all candidate actions fail verification, the process jumps to the classic controller takeover branch. Smooth takeover and control fusion: Based on the current mode and switching state, calculate the fusion weight of the world model control quantity and the classical controller control quantity, and obtain the final control quantity by weighting. Execution and Feedback Update: The final control quantity is sent to the UAV actuator for execution, and the actual status feedback after execution is collected to update the prediction reliability, historical prediction error and world model parameters before entering the next control cycle.

[0085] In the classical controller takeover branch, the classical flight controller outputs basic stability control commands, which are then executed after passing through the control fusion stage. Simultaneously, prediction reliability and safety status are continuously monitored. After several consecutive cycles of meeting the recovery conditions, the control weight of the world model is gradually increased, and the system recovers to a world model-dominated control mode with a delay. A hysteresis interval is set between the recovery threshold and the takeover threshold to avoid frequent switching near the threshold.

[0086] This embodiment fully covers the entire closed-loop process of UAV world model predictive control, organically combining dynamic imagination budget, safety gating, smooth takeover and hysteresis recovery mechanism, and realizing the collaborative work of intelligent prediction and safety backup.

[0087] In some embodiments, this embodiment addresses the issue of varying constraint importance under different flight scenarios by introducing an attention mechanism to dynamically adjust the verification weights and sorting priorities of each constraint dimension, thereby improving verification efficiency and task adaptability while ensuring safety.

[0088] The system pre-trains a scene attention network. Its input is the current scene feature vector, including five features: obstacle density, spatial complexity, task accuracy level, flight speed, and remaining battery power. The output is the attention weights for four dimensions: physical constraints, environmental constraints, task constraints, and actuator constraints, with the weights summing to 1. In each control cycle, the system first extracts the current scene features and inputs them into the attention network to obtain the weight allocation results for the four constraints.

[0089] During the verification process, a two-level verification mechanism of "hard constraint baseline + soft weight sorting" is adopted: Hard constraint bottom line layer: The core limit thresholds of the four dimensions still maintain a veto system. For example, the attitude angle limit in physical constraints, the obstacle intrusion judgment in environmental constraints, and the complete saturation in actuator constraints are all hard constraints that cannot be broken. As long as they are touched, they will be judged as unqualified. Soft constraint margin layer: For margin intervals that are close to the threshold but not exceeded, margin scores are calculated based on attention weights. For example, in dense obstacle scenarios, environmental constraints have the highest attention weight, and environmental margin scores account for a larger proportion under the same margin. Finally, the comprehensive margin score is used for the ranking and optimization of qualified sequences.

[0090] Meanwhile, the verification order is arranged from high to low attention weight, with the constraints with the highest weight being verified first. Unqualified sequences are terminated early, reducing the invalid calculation of low-weight constraints. For example, in open high-altitude flight scenarios, environmental constraints have the lowest weight and are verified last, allowing unqualified sequences to be screened out early through physical and actuator constraints.

[0091] This embodiment dynamically adapts to the constraint requirements of different flight scenarios through a scene attention mechanism, which not only preserves the security of hard constraints, but also improves the verification efficiency and scene adaptability, making it suitable for complex and ever-changing flight mission environments.

[0092] In some embodiments, this embodiment proposes an acceleration mechanism that combines incremental inference and step-by-step verification for embedded platforms with limited onboard computing resources, thereby reducing the computational latency and computing power consumption of security gating.

[0093] The traditional approach involves first generating a complete sequence of candidate actions, then deriving the entire time-domain trajectory, and finally performing a unified verification. This method suffers from high computational complexity and high latency. This embodiment employs an incremental process of "deriving one step, verifying one step, and terminating upon failure." The 0th step state of all candidate actions is initialized to the current real state of the drone, and all candidate sequences are marked as "to be verified" state; For all sequences to be verified, the control action and world model state at step k are deduced to obtain the predicted state at step k. The predicted state at step k is simultaneously subjected to constraint checks in four dimensions. If the step fails the check, the sequence is marked as "unqualified" and the deduction and check of all subsequent steps are terminated. Increment the k value by 1, and repeat steps 2-3 until all prediction time-domain steps are completed, or all candidate sequences are marked as unqualified.

[0094] For candidate sequences that terminate early, their remaining unanalyzed time-domain steps no longer consume computational resources of the world model. The saved computational power can be used to increase the number of optimization iterations for qualified sequences or improve the inference accuracy of the remaining sequences. At the same time, this mechanism can obtain unqualified results in advance, shortening the overall verification latency.

[0095] Tests showed that in obstacle-heavy scenarios, approximately 60% of candidate sequences can be eliminated in the first 5 steps, reducing overall verification latency by more than 40% and significantly improving the control frequency and response speed of low-computing-power platforms.

[0096] In some embodiments, this embodiment addresses multi-drone collaborative operation scenarios by incorporating multi-drone interval constraints into a security gating system to achieve distributed collaborative security verification.

[0097] In multi-UAV collaborative inspection and formation flight scenarios, each UAV, in addition to verifying its own four types of constraints, also needs to verify the safety distance constraints with other collaborative UAVs. In practice, each UAV periodically broadcasts its candidate action prediction trajectory and current flight status through an inter-UAV wireless communication link; each UAV receives the prediction trajectory data of other collaborative UAVs and incorporates it into the dynamic obstacle set in the environmental constraints.

[0098] During verification, a new inter-drone interval verification item is added to the environmental constraint dimension: the minimum distance between the predicted trajectory of the local drone and the predicted trajectories of all other drones is calculated step by step, and it is determined whether it is less than the preset inter-drone safety interval threshold (e.g., horizontal interval not less than 5m, altitude interval not less than 2m). If it is less than the threshold, the candidate action group is determined to have failed the environmental constraint verification.

[0099] For formation flight missions, a new formation maintenance check has been added to the mission constraint dimension: the deviation between the predicted trajectory and the predetermined position of the aircraft in the formation is calculated. If the deviation exceeds the formation maintenance accuracy threshold, the mission constraint check is deemed to have failed.

[0100] This embodiment extends single-machine safety gating to multi-machine collaborative scenarios, which can avoid the risk of collision in multi-machine operations, ensure the accuracy of formation, and is suitable for multiple UAV application scenarios such as cluster inspection and collaborative detection.

[0101] In some embodiments, this embodiment addresses the problem of actuator performance dynamically changing with battery voltage, temperature, and wear level by proposing an online actuator parameter identification mechanism to update the actuator constraint threshold in real time, thereby avoiding overload risks or performance losses caused by fixed thresholds.

[0102] The system has a built-in actuator parameter online identification module, which uses a recursive least squares algorithm with a forgetting factor. In each control cycle, it collects data on battery voltage, motor speed, acceleration feedback, and attitude angular velocity feedback to identify parameters such as motor thrust coefficient, torque coefficient, battery internal resistance, and servo response bandwidth online. It calculates the actual maximum output capability of the actuator under the current operating conditions in real time. For example, when the battery voltage drops, the maximum output thrust of the motor decreases synchronously. The system updates the maximum thrust threshold and the upper limit of motor speed in real time based on the identified current thrust coefficient and battery voltage.

[0103] During actuator constraint verification, a dynamically updated threshold, updated in real time, is used instead of a fixed threshold for saturation and abrupt change detection. A lower bound is also set to ensure that the identification result does not fall below the actuator's minimum rated performance.

[0104] This embodiment can adapt to the performance degradation and operating condition changes of the actuator, so that the constraint verification always matches the actual capability of the actuator. It avoids both being too conservative and wasting power performance under high voltage, and avoiding actuator overload and control failure caused by excessively high verification threshold under low voltage.

[0105] Please see Figure 3 As shown, Figure 3This is a schematic diagram of the structure of a UAV candidate action safety gating system 200 based on world model prediction provided in this application embodiment. The UAV candidate action safety gating system 200 based on world model prediction is used to execute the steps of the UAV candidate action safety gating method based on world model prediction shown in the above embodiments. The UAV candidate action safety gating system 200 based on world model prediction can be a single server or a server cluster, or it can be a terminal, such as a handheld terminal, laptop computer, wearable device, or robot.

[0106] like Figure 3 As shown, the UAV candidate action safety gating system 200 based on world model prediction includes: The sequence acquisition unit 201 is used to acquire multiple sets of candidate control action sequences generated by the world model, as well as the predicted flight trajectory data corresponding to each set of candidate actions; The constraint verification unit 202 is used to perform constraint verification on each group of candidate action sequences from preset dimensions. The constraint verification includes physical constraint verification to check whether the attitude angle, linear velocity and acceleration exceed the preset physical limits of the UAV; environmental constraint verification to check whether the predicted trajectory intrudes into the preset obstacle safety distance and no-fly zone; task constraint verification to check whether the trajectory meets the preset waypoint tracking and hovering accuracy requirements; and actuator constraint verification to check whether the control quantity causes actuator saturation or control quantity abrupt change. The preset dimensions include physical constraints, environmental constraints, task constraints and actuator constraints. The instruction output unit 203 is used to filter candidate control action sequences that pass the constraint verification as executable action outputs; if all candidate action sequences fail the safety gating verification, the preset classic flight controller is triggered to take over the flight control of the UAV and output basic stability control instructions.

[0107] In some embodiments, obtaining multiple sets of candidate control action sequences generated by the world model, and the predicted flight trajectory data corresponding to each set of candidate actions, includes: generating multiple sets of candidate control action sequences within the dynamic imagination budget limit of the current control cycle, each set of sequences containing control commands for multiple consecutive control cycles; inputting each set of candidate control action sequences into the world model to perform forward inference, and obtaining the predicted flight trajectory, attitude change sequence and risk assessment data corresponding to each set of actions.

[0108] In some embodiments, the physical constraint verification of attitude angle, linear velocity, and acceleration to see if they exceed preset UAV physical limits includes: extracting the predicted attitude angle, linear velocity, and acceleration values ​​corresponding to the candidate action sequence step by step; comparing each parameter with a preset UAV physical limit threshold one by one; if any parameter at any time exceeds the corresponding limit threshold, it is determined that the candidate action sequence has failed the physical constraint verification.

[0109] In some embodiments, the environmental constraint verification of whether the predicted trajectory intrudes into the preset obstacle safety distance and no-fly zone includes: constructing the spatial boundary between the obstacle-occupied area and the no-fly zone based on real-time environmental perception data; calculating the minimum distance between the predicted flight trajectory and each obstacle boundary at each sampling point, and determining whether the minimum distance is less than a preset safety distance threshold; determining whether the predicted trajectory enters the no-fly zone; if the trajectory intrudes into the safety distance or enters the no-fly zone, determining that the group of candidate action sequences has failed the environmental constraint verification.

[0110] In some embodiments, the process of verifying whether the mission constraint verification trajectory meets the preset waypoint tracking and hovering accuracy requirements includes: extracting the target waypoint sequence or hovering position coordinates in the current mission objective; calculating the tracking error between the predicted flight trajectory and each target waypoint, as well as the position deviation range during the hovering phase; if the tracking error or position deviation exceeds the preset accuracy threshold, determining that the corresponding candidate action sequence has failed the mission constraint verification.

[0111] In some embodiments, the actuator constraint verification of whether the control quantity causes actuator saturation or control quantity mutation includes: converting each control quantity in the candidate control action sequence into the output command of the corresponding actuator; checking whether the output command of each actuator exceeds its maximum output range to determine whether actuator saturation occurs; calculating the change amplitude of the control quantity between adjacent control cycles to determine whether it exceeds a preset mutation threshold; if actuator saturation or control quantity mutation exceeds the limit, determining that the corresponding candidate action sequence has failed the actuator constraint verification.

[0112] In some embodiments, the step of selecting candidate control action sequences that pass constraint verification as executable action output includes: after completing constraint verification for all candidate action sequences, collecting all candidate action sequences that pass all verifications; if there are multiple sets of sequences that pass verification, selecting the set with the highest comprehensive score as executable action output, wherein the comprehensive score is calculated based on trajectory tracking accuracy, energy consumption level and safety margin.

[0113] In some embodiments, after the candidate control action sequence that has passed the constraint verification is selected as the executable action output, the method further includes: continuously collecting feedback data on the actual flight state of the UAV during the execution of the executable action; comparing the actual state with the world model prediction state at the corresponding time and calculating the real-time execution deviation; if the execution deviation is within the allowable range, maintaining the current world model control mode; if the deviation continues to expand, dynamically shrinking the imagined budget scale of subsequent control cycles.

[0114] In some embodiments, the step of triggering a preset classic flight controller to take over the UAV flight control and output basic stability control commands if all candidate action sequences fail the safety gating verification includes: generating a mode switching trigger signal when all candidate action sequences fail the safety gating verification; gradually reducing the output weight of the world model control quantity and simultaneously increasing the output weight of the classic flight controller control quantity; after a preset transition period, completely switching to the classic flight controller to output control commands, and having the classic controller execute basic stability flight actions such as hovering, deceleration, or return to home.

[0115] It should be noted that those skilled in the art will understand that, for the sake of convenience and brevity, the specific working process of the UAV candidate action safety gating system and its modules based on world model prediction described above can be referred to the corresponding content in the various embodiments of the UAV candidate action safety gating method based on world model prediction, and will not be repeated here.

[0116] The aforementioned safety gating method for UAV candidate actions based on world model prediction can be implemented as a computer program, which can be used in, for example... Figure 3 It runs on the system shown.

[0117] Please see Figure 4 , Figure 4 This is a schematic block diagram of the structure of an autonomous flight control system for unmanned aerial vehicles (UAVs) provided in an embodiment of this application. The autonomous flight control system includes a processor, a memory, and a network interface connected via a device bus. The memory may include a storage medium and internal memory.

[0118] The storage medium may store operating devices and computer programs. The computer program includes program instructions that, when executed, cause the processor to perform any safety gating method for candidate drone actions based on world model predictions.

[0119] The processor provides computing and control capabilities to support the operation of the entire UAV autonomous flight control system.

[0120] The internal memory provides an environment for the execution of computer programs in non-volatile storage media. When the computer program is executed by the processor, it enables the processor to execute any safety gating method for candidate drone actions based on world model predictions.

[0121] This network interface is used for network communication, such as sending assigned tasks. Those skilled in the art will understand that... Figure 4 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the terminal to which the present application is applied. A specific UAV autonomous flight control system may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0122] It should be understood that the processor can be a Central Processing Unit (CPU), but it can also be other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. Among these, a general-purpose processor can be a microprocessor or any conventional processor.

[0123] In one embodiment, the processor is configured to run a computer program stored in memory to perform the following steps: Obtain multiple sets of candidate control action sequences generated by the world model, as well as the predicted flight trajectory data corresponding to each set of candidate actions; Each candidate action sequence is constrained and verified from preset dimensions. The constraint verification includes physical constraint verification to check whether the attitude angle, linear velocity and acceleration exceed the preset physical limits of the UAV; environmental constraint verification to check whether the predicted trajectory intrudes into the preset obstacle safety distance and no-fly zone; task constraint verification to check whether the trajectory meets the preset waypoint tracking and hovering accuracy requirements; and actuator constraint verification to check whether the control quantity causes actuator saturation or sudden change in control quantity. The preset dimensions include physical constraints, environmental constraints, task constraints and actuator constraints. Candidate control action sequences that pass the constraint verification are selected as executable action outputs; if all candidate action sequences fail the safety gating verification, the preset classic flight controller is triggered to take over the UAV flight control and output basic stability control commands.

[0124] In some embodiments, obtaining multiple sets of candidate control action sequences generated by the world model, and the predicted flight trajectory data corresponding to each set of candidate actions, includes: generating multiple sets of candidate control action sequences within the dynamic imagination budget limit of the current control cycle, each set of sequences containing control commands for multiple consecutive control cycles; inputting each set of candidate control action sequences into the world model to perform forward inference, and obtaining the predicted flight trajectory, attitude change sequence and risk assessment data corresponding to each set of actions.

[0125] In some embodiments, the physical constraint verification of attitude angle, linear velocity, and acceleration to see if they exceed preset UAV physical limits includes: extracting the predicted attitude angle, linear velocity, and acceleration values ​​corresponding to the candidate action sequence step by step; comparing each parameter with a preset UAV physical limit threshold one by one; if any parameter at any time exceeds the corresponding limit threshold, it is determined that the candidate action sequence has failed the physical constraint verification.

[0126] In some embodiments, the environmental constraint verification of whether the predicted trajectory intrudes into the preset obstacle safety distance and no-fly zone includes: constructing the spatial boundary between the obstacle-occupied area and the no-fly zone based on real-time environmental perception data; calculating the minimum distance between the predicted flight trajectory and each obstacle boundary at each sampling point, and determining whether the minimum distance is less than a preset safety distance threshold; determining whether the predicted trajectory enters the no-fly zone; if the trajectory intrudes into the safety distance or enters the no-fly zone, determining that the group of candidate action sequences has failed the environmental constraint verification.

[0127] In some embodiments, the process of verifying whether the mission constraint verification trajectory meets the preset waypoint tracking and hovering accuracy requirements includes: extracting the target waypoint sequence or hovering position coordinates in the current mission objective; calculating the tracking error between the predicted flight trajectory and each target waypoint, as well as the position deviation range during the hovering phase; if the tracking error or position deviation exceeds the preset accuracy threshold, determining that the corresponding candidate action sequence has failed the mission constraint verification.

[0128] In some embodiments, the actuator constraint verification of whether the control quantity causes actuator saturation or control quantity mutation includes: converting each control quantity in the candidate control action sequence into the output command of the corresponding actuator; checking whether the output command of each actuator exceeds its maximum output range to determine whether actuator saturation occurs; calculating the change amplitude of the control quantity between adjacent control cycles to determine whether it exceeds a preset mutation threshold; if actuator saturation or control quantity mutation exceeds the limit, determining that the corresponding candidate action sequence has failed the actuator constraint verification.

[0129] In some embodiments, the step of selecting candidate control action sequences that pass constraint verification as executable action output includes: after completing constraint verification for all candidate action sequences, collecting all candidate action sequences that pass all verifications; if there are multiple sets of sequences that pass verification, selecting the set with the highest comprehensive score as executable action output, wherein the comprehensive score is calculated based on trajectory tracking accuracy, energy consumption level and safety margin.

[0130] In some embodiments, after the candidate control action sequence that has passed the constraint verification is selected as the executable action output, the method further includes: continuously collecting feedback data on the actual flight state of the UAV during the execution of the executable action; comparing the actual state with the world model prediction state at the corresponding time and calculating the real-time execution deviation; if the execution deviation is within the allowable range, maintaining the current world model control mode; if the deviation continues to expand, dynamically shrinking the imagined budget scale of subsequent control cycles.

[0131] In some embodiments, the step of triggering a preset classic flight controller to take over the UAV flight control and output basic stability control commands if all candidate action sequences fail the safety gating verification includes: generating a mode switching trigger signal when all candidate action sequences fail the safety gating verification; gradually reducing the output weight of the world model control quantity and simultaneously increasing the output weight of the classic flight controller control quantity; after a preset transition period, completely switching to the classic flight controller to output control commands, and having the classic controller execute basic stability flight actions such as hovering, deceleration, or return to home.

[0132] This application also provides a computer-readable storage medium storing a computer program that, when executed by a processor, causes the processor to implement the steps of the safety gating method for candidate drone actions based on world model prediction provided in any embodiment of this application.

[0133] The computer-readable storage medium can be an internal storage unit of the UAV autonomous flight control system described in the foregoing embodiments, such as the hard drive or memory of the UAV autonomous flight control system. Alternatively, the computer-readable storage medium can be an external storage device of the UAV autonomous flight control system, such as a plug-in hard drive, Smart Media Card (SMC), Secure Digital (SD) card, or Flash Card equipped on the UAV autonomous flight control system.

[0134] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any person skilled in the art can easily conceive of various equivalent modifications or substitutions within the technical scope disclosed in this application, and these modifications or substitutions should all be covered within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A safety gating method for candidate actions of unmanned aerial vehicles (UAVs) based on world model prediction, applied to an autonomous flight control system for UAVs equipped with a world model prediction module, characterized in that, include: Obtain multiple sets of candidate control action sequences generated by the world model, as well as the predicted flight trajectory data corresponding to each set of candidate actions; Each candidate action sequence is constrained and verified from preset dimensions. The constraint verification includes physical constraint verification to check whether the attitude angle, linear velocity and acceleration exceed the preset physical limits of the UAV; environmental constraint verification to check whether the predicted trajectory intrudes into the preset obstacle safety distance and no-fly zone; task constraint verification to check whether the trajectory meets the preset waypoint tracking and hovering accuracy requirements; and actuator constraint verification to check whether the control quantity causes actuator saturation or sudden change in control quantity. The preset dimensions include physical constraints, environmental constraints, task constraints and actuator constraints. Candidate control action sequences that pass the constraint verification are selected as executable action outputs; if all candidate action sequences fail the safety gating verification, the preset classic flight controller is triggered to take over the UAV flight control and output basic stability control commands.

2. The method according to claim 1, characterized in that, The acquisition of multiple sets of candidate control action sequences generated by the world model, and the predicted flight trajectory data corresponding to each set of candidate actions, includes: Within the dynamic imagination budget limit of the current control cycle, multiple sets of candidate control action sequences are generated, each set of sequences containing control instructions for multiple consecutive control cycles; Each set of candidate control action sequences is input into the world model to perform forward extrapolation, resulting in the predicted flight trajectory, attitude change sequence, and risk assessment data for each set of actions.

3. The method according to claim 1, characterized in that, The physical constraint verification checks whether the attitude angle, linear velocity, and acceleration exceed the preset physical limits of the UAV, including: Extract the predicted attitude angle, linear velocity, and acceleration values ​​corresponding to the candidate action sequence step by step; Each parameter is compared with the preset physical limit threshold of the drone one by one; If any parameter at any time exceeds the corresponding limit threshold, the candidate action sequence is deemed to have failed the physical constraint verification.

4. The method according to claim 1, characterized in that, The environmental constraint verification checks whether the predicted trajectory intrudes into the preset obstacle safety distance and no-fly zone, including: The spatial boundaries between the obstacle-occupied area and the no-fly zone are constructed based on real-time environmental perception data; The minimum distance between the predicted flight trajectory and the boundary of each obstacle is calculated for each sampling point, and it is determined whether the minimum distance is less than the preset safe distance threshold. Determine whether the predicted trajectory enters the no-fly zone; if the trajectory intrudes into the safe distance or enters the no-fly zone, determine that the candidate action sequence has failed the environmental constraint verification.

5. The method according to claim 1, characterized in that, Whether the task constraint verification trajectory meets the preset waypoint tracking and hovering accuracy requirements includes: Extract the target waypoint sequence or hovering position coordinates from the current mission objective; Calculate the tracking error between the predicted flight trajectory and each target waypoint, as well as the position deviation range during the hovering phase; If the tracking error or position deviation exceeds the preset accuracy threshold, the corresponding candidate action sequence is determined to have failed the task constraint verification.

6. The method according to claim 1, characterized in that, The actuator constraint verification check determines whether the control quantity causes actuator saturation or a sudden change in control quantity, including: Convert each control variable in the candidate control action sequence into the output command of the corresponding actuator; Check whether the output command of each actuator exceeds its maximum output range to determine whether the actuator saturation state has occurred. Calculate the change in control quantity between adjacent control cycles and determine whether it exceeds the preset sudden change threshold; If actuator saturation or control variable abruptly exceeds the limit, the corresponding candidate action sequence is determined to have failed the actuator constraint check.

7. The method according to claim 1, characterized in that, The candidate control action sequences that pass the constraint verification are output as executable actions, including: After completing constraint verification for all candidate action sequences, collect all candidate action sequences that have passed all verifications. If multiple sequences pass the verification, the one with the highest comprehensive score is selected as the executable action output. The comprehensive score is calculated based on trajectory tracking accuracy, energy consumption level, and safety margin.

8. The method according to claim 1, characterized in that, After the candidate control action sequences that pass the constraint verification are output as executable actions, the following steps are also included: During the execution of executable actions, the drone's real flight status feedback data is continuously collected; The actual state is compared with the world model prediction state at the corresponding moment to calculate the real-time execution deviation; If the deviation is within the allowable range, maintain the current world model control mode; if the deviation continues to expand, dynamically shrink the imagined budget size for subsequent control cycles.

9. The method according to claim 1, characterized in that, If all candidate action sequences fail the security gating check, the preset classic flight controller is triggered to take over the UAV flight control and output basic stability control commands, including: When all candidate action sequences fail the security gating check, a mode switching trigger signal is generated. Gradually reduce the output weight of the world model control variables, while simultaneously increasing the output weight of the classic flight controller control variables; After a preset transition period, the system fully switches to the classic flight controller to output control commands, and the classic controller executes basic stable flight maneuvers such as hovering, deceleration, or returning to home.

10. A safety gating system for candidate actions of a UAV based on world model prediction, applied to an autonomous flight control system of a UAV equipped with a world model prediction module, for implementing the method as described in any one of claims 1-9, characterized in that, include: The sequence acquisition unit is used to acquire multiple sets of candidate control action sequences generated by the world model, as well as the predicted flight trajectory data corresponding to each set of candidate actions; The constraint verification unit is used to perform constraint verification on each group of candidate action sequences from preset dimensions. The constraint verification includes physical constraint verification to check whether the attitude angle, linear velocity and acceleration exceed the preset physical limits of the UAV; environmental constraint verification to check whether the predicted trajectory intrudes into the preset obstacle safety distance and no-fly zone; task constraint verification to check whether the trajectory meets the preset waypoint tracking and hovering accuracy requirements; and actuator constraint verification to check whether the control quantity causes actuator saturation or control quantity abrupt change. The preset dimensions include physical constraints, environmental constraints, task constraints and actuator constraints. The instruction output unit is used to filter candidate control action sequences that pass the constraint verification as executable action outputs; if all candidate action sequences fail the safety gating verification, the preset classic flight controller is triggered to take over the UAV flight control and output basic stability control instructions.

Images