Unmanned aerial vehicle control method and system based on autonomous perception and decision making using perception models

Abstract

The application belongs to the technical field of unmanned aerial vehicle flight control, and relates to an unmanned aerial vehicle control method and system based on autonomous perception and decision-making based on a perception model, comprising: acquiring task instructions, environment perception data and flight state data, performing time synchronization, coordinate conversion and unified feature coding on multi-source data to generate unified multi-modal features, inputting the unified multi-modal features and the task instructions into a perception large model to obtain a structured scene state, generating candidate decisions according to the structured scene state, the task target and the flight state data, performing safety constraint verification on the candidate decisions to obtain effective decisions, converting the effective decisions into flight control instructions and controlling flight, and updating the structured scene state according to flight feedback data. The application solves the problems that multi-sensor data is difficult to form unified understanding and that a large model output directly enters a flight control, thereby improving the safety and task execution capability of autonomous flight of the unmanned aerial vehicle.

Description

Technical Field

[0001] This application relates to the field of unmanned aerial vehicle (UAV) flight control technology, and in particular to a UAV control method and system based on autonomous perception and decision-making using a perception model. Background Technology

[0002] Existing autonomous flight systems for unmanned aerial vehicles (UAVs) typically acquire environmental information and flight status through cameras, lidar, millimeter-wave radar, inertial measurement units, and satellite positioning modules, and then utilize target detection, localization and mapping, path planning, and flight control algorithms to achieve autonomous flight. Some solutions also utilize visual language models or large language models to parse mission instructions and generate waypoints or action commands.

[0003] In existing solutions, data from different sensors are typically processed by independent algorithms, making it difficult to form a unified understanding of environmental objects, spatial relationships, motion trends, and mission risks. Control methods based on fixed rules have limited adaptability to unknown obstacles, dynamic targets, and complex natural language tasks. Furthermore, large model outputs may suffer from delays, insufficient confidence levels, or unexecutable commands. If directly sent to the flight controller, these can easily lead to abrupt trajectory changes, collisions, out-of-bounds flight, or loss of control. Therefore, how to integrate multi-source perception information to form a unified environmental understanding, and how to utilize a large perception model to autonomously generate decisions while performing safety constraint verification before execution, are the technical problems that need to be solved.

[0004] In view of the above, this application is hereby submitted. Summary of the Invention

[0005] The purpose of this application is to propose a method and system for controlling unmanned aerial vehicles (UAVs) based on autonomous perception and decision-making using a perception model. The aim is to integrate multi-source perception information to form a unified environmental understanding, and to autonomously generate decisions using a large perception model while performing safety constraint verification before execution.

[0006] To address the aforementioned technical problems, this application provides a UAV control method based on autonomous perception and decision-making using a perception model, employing the following technical solution: Acquire mission instructions, environmental awareness data, and flight status data; Time synchronization and coordinate transformation are performed on environmental perception data and flight status data to obtain synchronized data, and feature encoding is performed on the synchronized data to generate unified multimodal features; By inputting unified multimodal features and task instructions into the perception big model, a structured scene state is obtained; Based on the structured scenario state, the mission objectives corresponding to the mission instructions, and flight status data, candidate decisions are generated; Perform security constraint verification on candidate decisions to obtain valid decisions that pass the verification. Effective decisions are translated into flight control commands, flight is controlled according to the flight control commands, and the structured scenario state is updated based on flight feedback data.

[0007] Furthermore, time synchronization and coordinate transformation are performed on environmental perception data and flight status data to obtain synchronized data. Feature encoding is then performed on the synchronized data to generate unified multimodal features, including: Denoising and time-aligning are performed on environmental perception data and flight status data; Based on the sensor installation parameters and the drone's motion status, coordinate transformation and motion compensation are performed on the time-aligned data to obtain synchronized data. Extract the features corresponding to each data type in the synchronized data; The extracted features are appended with time identifiers, spatial location identifiers, sensor source identifiers, and confidence identifiers to obtain unified multimodal features.

[0008] Furthermore, by inputting unified multimodal features and task instructions into the large perception model, a structured scene state is obtained, including: By using a large perception model to jointly reason about unified multimodal features and mission instructions, the system determines the category, location, speed, direction of motion of environmental objects and their spatial relationship with the UAV, and identifies passable areas, obstacle areas, mission objectives and risk areas. Transform the reasoning results into a structured scene state in the form of a scene graph, semantic map, object list, or state vector.

[0009] Furthermore, by inputting unified multimodal features and task instructions into the large perception model, a structured scene state is obtained, including: Obtain historical scene status; The structured scene state output by the perception model is fused with the historical scene state to update the structured scene state.

[0010] Furthermore, security constraints are validated on the candidate decisions to obtain valid decisions that pass the validation, including: Obtain safety constraints, which include at least one of the following: maximum speed, maximum acceleration, flight altitude range, geofence, minimum safe distance from obstacles, remaining battery threshold, location reliability threshold, and communication quality threshold. Candidate decisions are validated based on safety constraints, and those that pass the validation are considered valid decisions.

[0011] Furthermore, candidate decisions are validated based on safety constraints, and those that pass the validation are considered valid decisions, including: When there are candidate decisions that do not meet the safety constraints, the candidate decisions that do not meet the constraints are rejected, the speed is reduced, the obstacle avoidance distance is increased, or the plan is redesigned.

[0012] Furthermore, the structured scenario status is updated based on flight feedback data, including: The deviation between the actual flight results and the predicted results of the candidate decisions is determined based on flight feedback data; When the deviation exceeds the preset threshold, the steps of acquiring mission instructions, environmental perception data, and flight status data are re-executed until the structured scene status is updated based on the flight feedback data.

[0013] Furthermore, after inputting unified multimodal features and task instructions into the large perception model to obtain the structured scene state, it includes: Monitor the inference time, output confidence, continuous output consistency, communication status, and sensor status of the large-scale perception model; When the inference time exceeds the preset time threshold, the output confidence is lower than the preset confidence threshold, the continuous outputs conflict with each other, the communication is interrupted, or the key sensor is abnormal, the newly generated candidate decisions of the perception model are stopped from being used to generate effective decisions, and the system switches to the local lightweight model, preset rule control, hovering, return, landing, or manual remote control mode.

[0014] Furthermore, based on the structured scenario state, the mission objectives corresponding to the mission instructions, and flight status data, candidate decisions are generated, including: Determine the current task phase; Based on the structured scenario state and flight state data, generate at least one candidate action or candidate trajectory corresponding to the current mission stage and mission objective. The candidate decision includes the candidate action or candidate trajectory and the corresponding decision confidence level. The candidate decisions are evaluated based on task completion, collision risk, energy consumption, trajectory smoothness, and communication status. Based on the evaluation results, the candidate decisions used for safety constraint verification are selected from the candidate decisions.

[0015] To address the aforementioned technical problems, this application also provides a drone control system based on a perception model for autonomous perception and decision-making, employing the following technical solution: A drone control system based on a perception model for autonomous perception and decision-making includes: The data acquisition module is used to acquire mission instructions, environmental perception data, and flight status data; The feature generation module is used to synchronize environmental perception data and flight status data in time and transform coordinates to obtain synchronized data, and to encode the synchronized data to generate unified multimodal features. The scene generation module is used to input unified multimodal features and task instructions into the perception big model to obtain a structured scene state; The decision generation module is used to generate candidate decisions based on the structured scenario state, the mission objectives corresponding to the mission instructions, and flight status data. The security verification module is used to perform security constraint verification on candidate decisions and obtain valid decisions that pass the verification. The execution feedback module is used to convert effective decisions into flight control commands, control flight according to flight control commands, and update the structured scenario state based on flight feedback data.

[0016] To address the aforementioned technical problems, this application also provides a computer device that employs the following technical solution: A computer device includes a memory and a processor, wherein the memory stores computer-readable instructions, and the processor executes the computer-readable instructions to implement the steps of the UAV control method based on a perception model for autonomous perception and decision-making as described above.

[0017] To address the aforementioned technical problems, this application also provides a computer-readable storage medium, employing the technical solution described below: A computer-readable storage medium storing computer-readable instructions, which, when executed by a processor, implement the steps of the UAV control method based on a perception model for autonomous perception and decision-making as described above.

[0018] Compared with the prior art, this application has the following main advantages: The UAV control method disclosed in this application, which is based on autonomous perception and decision-making using a perception model, acquires mission instructions, environmental perception data, and flight status data, providing a complete raw data foundation for subsequent unified perception and mission understanding.

[0019] By synchronizing environmental perception data and flight status data in time and transforming coordinates, synchronized data is obtained. The synchronized data is then feature-encoded to generate unified multimodal features. Multi-source heterogeneous sensor data is transformed into a unified feature representation with spatiotemporal source and credibility information, enabling the perception big model to perform cross-modal reasoning in a unified semantic space. This solves the problem in existing technologies where different sensor data are processed independently and it is difficult to form a unified understanding.

[0020] By inputting unified multimodal features and task instructions into the perception big model, a structured scene state is obtained; the system acquires structured semantic information including environmental objects, spatial relationships, task objectives and risk areas, providing a complete scene cognition foundation for decision generation.

[0021] By generating candidate decisions based on structured scenario states, mission objectives corresponding to mission instructions, and flight status data, the decision generation process takes into account mission requirements, environmental constraints, and flight status, thereby improving the relevance and feasibility of the decisions.

[0022] By performing safety constraint verification on candidate decisions, valid decisions that pass the verification are obtained; an independent safety barrier is established between the output of the perception model and the flight controller to prevent untrusted decisions from directly entering the flight control system, thus solving the problems of collision, boundary crossing and loss of control caused by large model output delay, insufficient confidence or inability to execute.

[0023] By converting effective decisions into flight control commands, controlling flight according to flight control commands, and updating the structured scenario state based on flight feedback data, a closed loop of perception, decision-making, verification, execution, and feedback is formed. This enables the system to correct execution deviations in real time and adapt to dynamic environmental changes, thereby improving the safety, stability, and mission execution capabilities of UAV autonomous flight. Attached Figure Description

[0024] To more clearly illustrate the solutions in this application, the accompanying drawings used in the description of the embodiments of this application will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0025] Figure 1 This is an exemplary system architecture diagram to which this application can be applied; Figure 2 This is a flowchart of an embodiment of the UAV control method based on a perception model for autonomous perception and decision-making according to this application; Figure 3 This is a schematic diagram of an embodiment of an unmanned aerial vehicle (UAV) control system based on a perception model for autonomous perception and decision-making according to this application; Figure 4 This is a schematic diagram of the structure of one embodiment of the computer device according to this application. Detailed Implementation

[0026] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application pertains; the terminology used herein in the specification of the application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application; the terms "comprising" and "having," and any variations thereof, in the specification, claims, and foregoing drawings of this application, are intended to cover non-exclusive inclusion. The terms "first," "second," etc., in the specification, claims, or foregoing drawings of this application are used to distinguish different objects, not to describe a particular order.

[0027] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of this application. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a mutually exclusive, independent, or alternative embodiment. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0028] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings.

[0029] like Figure 1 As shown, the system architecture 100 may include a first terminal device 101, a second terminal device 102, a third terminal device 103, a network 104, and a server 105. The network 104 serves as a medium for providing communication links between the first terminal device 101, the second terminal device 102, the third terminal device 103, and the server 105. The network 104 may include various connection types, such as wired or wireless communication links, or fiber optic cables, etc.

[0030] Users can use the first terminal device 101, the second terminal device 102, and the third terminal device 103 to interact with the server 105 via the network 104 to receive or send messages, etc. Various communication client applications can be installed on the first terminal device 101, the second terminal device 102, and the third terminal device 103, such as web browser applications, shopping applications, search applications, instant messaging tools, email clients, social platform software, etc.

[0031] The first terminal device 101, the second terminal device 102, and the third terminal device 103 can be various electronic devices with displays and support web browsing, including but not limited to smartphones, tablets, e-book readers, MP3 (Moving Picture Experts Group Audio Layer III) players, MP4 (Moving Picture Experts Group Audio Layer IV) players, laptops, and desktop computers, etc.

[0032] Server 105 can be a server that provides various services, such as a backend server that supports the pages displayed on the first terminal device 101, the second terminal device 102, and the third terminal device 103.

[0033] It should be noted that the UAV control method based on autonomous perception and decision-making based on the perception model provided in this application is generally executed by the terminal device, and correspondingly, the UAV control system based on autonomous perception and decision-making based on the perception model is generally set in the terminal device.

[0034] It should be understood that Figure 1 The number of terminal devices, networks, and servers shown is merely illustrative. Depending on implementation needs, any number of terminal devices, networks, and servers can be included.

[0035] Continue to refer to Figure 2 The diagram illustrates a flowchart of an embodiment of the UAV control method based on a perception model for autonomous perception and decision-making according to this application. The UAV control method based on a perception model for autonomous perception and decision-making includes the following steps: Step S201: Acquire mission instructions, environmental perception data, and flight status data.

[0036] In this embodiment, the UAV control method based on autonomous perception and decision-making using a perception model operates on electronic devices (e.g., Figure 1 The terminal device shown can send or receive data via wired or wireless connection. It should be noted that the aforementioned wireless connection methods may include, but are not limited to, 3G / 4G / 5G connections, Wi-Fi connections, Bluetooth connections, WiMAX connections, Zigbee connections, UWB (ultra-wide band) connections, and other currently known or future wireless connection methods.

[0037] Mission commands refer to high-level intent descriptions that trigger the UAV to perform specific flight missions. These commands can take the form of natural language commands, structured mission commands, or pre-defined waypoint missions. Environmental perception data refers to raw or pre-processed information collected by the UAV from its surrounding environment using various onboard sensors, including at least two of the following: images, depth information, point clouds, radar range, and target velocity information. Flight status data refers to information describing the UAV's own motion and system status, including at least one of the following: position, altitude, attitude, speed, heading, remaining battery power, communication status, and historical trajectory.

[0038] In this embodiment, the three types of data mentioned above are read from the sensor driver layer, the mission receiving interface, and the flight control status interface, respectively, and sent to a unified data buffer queue to provide raw input for subsequent time alignment and feature encoding. By synchronously acquiring multi-dimensional information, it is ensured that the subsequent understanding of the environment and mission is based on complete data.

[0039] Step S202: Time synchronization and coordinate transformation are performed on the environmental perception data and flight status data to obtain synchronized data, and feature encoding is performed on the synchronized data to generate unified multimodal features.

[0040] Unified multimodal features refer to a unified feature representation formed by aligning and encoding data from different sensors, with different physical meanings and timestamps, and adding spatiotemporal source and credibility information.

[0041] In this embodiment, the environmental perception data and flight status data are first denoised and time-aligned. Time interpolation is performed based on the timestamps carried in each data frame to align all data to the same decision time. Then, the time-aligned data is subjected to coordinate transformation and motion compensation using pre-calibrated sensor installation parameters and the current motion state of the UAV, unifying the sensor data into the same coordinate system to obtain synchronized data. Next, the corresponding feature extraction encoder is called according to the data type to extract the features corresponding to each data type. Finally, time identifiers, spatial location identifiers, sensor source identifiers, and confidence identifiers are added to the extracted features to generate unified multimodal features.

[0042] Step S203: Input the unified multimodal features and task instructions into the perception big model to obtain the structured scene state.

[0043] A large-scale perceptual model refers to a neural network model that has been pre-trained on a large scale and possesses multimodal understanding and reasoning capabilities. It can simultaneously process visual, spatial, and textual information and map it to a unified semantic space. A structured scene state refers to a structured description of information such as environmental objects, task elements, spatial relationships, and risks. Its forms include scene graphs, semantic maps, object lists, or state vectors.

[0044] In this embodiment, the unified multimodal features and task instructions generated in step S202 are jointly input into the perception big model. The perception big model performs joint reasoning on the multimodal information through an internal cross-modal attention mechanism to determine the category, position, speed, direction of motion of environmental objects and their spatial relationship with the UAV, and identifies passable areas, obstacle areas, task objectives and risk areas. Then, the reasoning results are converted into structured scene states in the form of scene graphs, semantic maps, object lists or state vectors.

[0045] Step S204: Generate candidate decisions based on the structured scenario state, the mission objective corresponding to the mission instruction, and the flight status data.

[0046] The mission objective is a specific sub-objective that the UAV needs to achieve, parsed from the mission instructions. Candidate decisions refer to possible sequences of actions or trajectory schemes generated to achieve the mission objective. Each candidate decision includes action type, target position, target velocity, target altitude, target heading, effective time, and decision confidence.

[0047] In this embodiment, the current task stage is determined based on the structured scene state. Based on the task stage, task objective and current flight state, at least one candidate action or candidate trajectory is generated using the decision module embedded in the perception big model or an independent planning model, and the confidence level of each candidate decision is given.

[0048] Step S205: Perform security constraint verification on the candidate decisions to obtain valid decisions that pass the verification.

[0049] Safety constraint verification refers to evaluating candidate decisions one by one based on pre-defined flight safety boundary conditions, and only allowing decisions that meet all safety conditions. Effective decisions are those that have been verified as safe and feasible and can be directly used to generate flight control commands.

[0050] In this embodiment, preset safety constraints are obtained, and each candidate decision is checked in turn to see if it violates any constraint. Candidate decisions that pass all constraint checks are marked as valid decisions. For candidate decisions that do not meet the constraints, the following actions are taken: rejection, speed reduction, increasing obstacle avoidance distance, or replanning. If all candidate decisions fail the checks, hovering, return, or landing commands are generated according to the preset safety strategy as valid decisions.

[0051] Step S206: Convert the effective decision into flight control commands, control the flight according to the flight control commands, and update the structured scenario state according to the flight feedback data.

[0052] Flight control commands are the low-level control quantities that the flight controller can directly parse and execute, including waypoint position, desired velocity vector, and desired attitude angle or angular velocity. Flight feedback data refers to the actual position, velocity, attitude, trajectory deviation, and new environmental perception data continuously acquired during flight.

[0053] In this embodiment, the effective decision obtained in step S205 is mapped into flight control commands for the corresponding control channel through the command conversion module and sent to the flight controller to drive the actuator. At the same time, flight feedback data is continuously acquired, and the deviation between the actual flight result and the candidate decision prediction result is determined based on the flight feedback data. When the deviation exceeds a preset threshold, the complete closed loop of S201 to S206 is re-executed to update the structured scene state.

[0054] This application acquires mission instructions, environmental perception data, and flight status data; providing a complete raw data foundation for subsequent unified perception and mission understanding.

[0055] By synchronizing environmental perception data and flight status data in time and transforming coordinates, synchronized data is obtained. The synchronized data is then feature-encoded to generate unified multimodal features. Multi-source heterogeneous sensor data is transformed into a unified feature representation with spatiotemporal source and credibility information, enabling the perception big model to perform cross-modal reasoning in a unified semantic space. This solves the problem in existing technologies where different sensor data are processed independently and it is difficult to form a unified understanding.

[0056] By inputting unified multimodal features and task instructions into the perception big model, a structured scene state is obtained; the system acquires structured semantic information including environmental objects, spatial relationships, task objectives and risk areas, providing a complete scene cognition foundation for decision generation.

[0057] By generating candidate decisions based on structured scenario states, mission objectives corresponding to mission instructions, and flight status data, the decision generation process takes into account mission requirements, environmental constraints, and flight status, thereby improving the relevance and feasibility of the decisions.

[0058] By performing safety constraint verification on candidate decisions, valid decisions that pass the verification are obtained; an independent safety barrier is established between the output of the perception model and the flight controller to prevent untrusted decisions from directly entering the flight control system, thus solving the problems of collision, boundary crossing and loss of control caused by large model output delay, insufficient confidence or inability to execute.

[0059] By converting effective decisions into flight control commands, controlling flight according to flight control commands, and updating the structured scenario state based on flight feedback data, a closed loop of perception, decision-making, verification, execution, and feedback is formed. This enables the system to correct execution deviations in real time and adapt to dynamic environmental changes, thereby improving the safety, stability, and mission execution capabilities of UAV autonomous flight.

[0060] In some optional implementations of this embodiment, the steps of synchronizing environmental perception data and flight status data in time and transforming coordinates to obtain synchronized data, and then encoding the synchronized data to generate unified multimodal features include: Denoising and time-aligning are performed on environmental perception data and flight status data; Based on the sensor installation parameters and the drone's motion status, coordinate transformation and motion compensation are performed on the time-aligned data to obtain synchronized data. Extract the features corresponding to each data type in the synchronized data; The extracted features are appended with time identifiers, spatial location identifiers, sensor source identifiers, and confidence identifiers to obtain unified multimodal features.

[0061] Noise reduction refers to filtering out random noise and outliers from the raw sensor signal. Time alignment refers to mapping data frames with different timestamps to a unified time reference through interpolation. Coordinate transformation refers to transforming the coordinates of various sensors to the same reference coordinate system using the sensor extrinsic parameter matrix. Motion compensation refers to eliminating motion distortion in sensor data based on the UAV's motion state. Time stamp refers to a marker recording the data acquisition time corresponding to a feature. Spatial location stamp refers to a marker recording the spatial location corresponding to a feature. Sensor source stamp refers to a marker indicating the type of sensor from which the feature originated. Confidence stamp refers to a marker indicating the degree of confidence in the feature.

[0062] In this embodiment, noise reduction employs sliding window mid-range filtering or Kalman filtering; time alignment maps each data frame to a unified time reference using linear interpolation or spline interpolation; coordinate transformation utilizes the extrinsic parameter matrices from each sensor to the UAV's body coordinate system for rigid body transformation; motion compensation, based on the angular and linear velocities measured by the inertial measurement unit, compensates data collected at different times to the same reference pose, eliminating motion distortion and thus obtaining synchronized data. During feature extraction, a corresponding feature extraction encoder is selected based on the data type. This encoder is a neural network module that transforms raw data into fixed-dimensional feature vectors; for example, convolutional neural networks are used for visual data, point cloud networks for point cloud data, and multilayer perceptrons are used to encode state vectors. Subsequently, time, spatial location, sensor source, and confidence level identifiers are added to the extracted features, with the confidence level identifier determined based on the sensor noise model or calibration accuracy. The features with added identifiers are packaged into unified multimodal features and input into the subsequent large-scale perception model for processing.

[0063] This application unifies the encoding of multi-sensor data after denoising, time alignment, coordinate transformation, and motion compensation, and explicitly adds spatiotemporal source and credibility identifiers. This enables sensor data with different physical meanings to have a clear reference benchmark in a unified semantic space, eliminating perceptual ambiguity caused by data misalignment and improving the accuracy and robustness of subsequent environmental understanding.

[0064] In some optional implementations of this embodiment, the steps described above for unifying multimodal features and task instruction input perception models to obtain structured scene states include: By using a large perception model to jointly reason about unified multimodal features and mission instructions, the system determines the category, location, speed, direction of motion of environmental objects and their spatial relationship with the UAV, and identifies passable areas, obstacle areas, mission objectives and risk areas. Transform the reasoning results into a structured scene state in the form of a scene graph, semantic map, object list, or state vector.

[0065] Joint reasoning refers to a large perceptual model simultaneously processing multimodal features and task instructions through a cross-modal attention mechanism, establishing connections between different pieces of information during feature interactions. A passable area refers to a spatial region that a drone can safely fly over. An obstacle region refers to a spatial region containing obstacles that need to be avoided. A risk region refers to a spatial region with potential dangers that require cautious flight. A scene graph is a graph structure representing objects and their relationships using nodes and edges. A semantic map is an occupied grid map with semantic labels. An object list is a list structure containing attribute information for each object. A state vector is a high-dimensional feature vector output by the model.

[0066] In this embodiment, the perception big data model adopts a Transformer architecture, processing unified multimodal feature sequences through a self-attention mechanism. It learns the spatial relationships, relative motions, and relevance to task commands between different objects during feature interactions, determining the category, location, speed, direction of motion of environmental objects, and their spatial relationship with the UAV. It also identifies traversable areas, obstacle areas, task objectives, and risk areas. The output inference results can be converted into structured scene states in the form of scene graphs, semantic maps, object lists, or state vectors, depending on application requirements. Structured scene states enable the decision model to directly perform logical judgments and numerical optimizations based on semantically clear and structurally regular information, avoiding repeated reprocessing of the original high-dimensional sensor data.

[0067] This application uses a large perceptual model to perform joint reasoning on multimodal features and task instructions, enabling the system not only to detect objects in the environment, but also to understand the relationship between each object and the current task. This achieves a leap from fragmented perception to unified understanding at the task level, providing a complete semantic foundation for subsequent decision-making.

[0068] In some optional implementations of this embodiment, the steps described above for unifying multimodal features and task instruction input perception models to obtain structured scene states include: Obtain historical scene status; The structured scene state output by the perception model is fused with the historical scene state to update the structured scene state.

[0069] Historical scene state refers to the structured scene state generated and stored by the perception model before the current moment. Fusion refers to combining the historical scene state with the current output at a probabilistic level to obtain a more stable and reliable state estimate.

[0070] In this embodiment, the historical scene state obtained from the previous one or several inference iterations is acquired and fused with the newly output structured scene state of the current perception model. The fusion is implemented using a Bayesian filtering framework. For dynamic objects, the historical state is used as the prior, and the current output as the observation; the motion state is updated using Kalman filtering. For static scenes, exponential moving average is used for smooth updates to eliminate single-frame noise. For grid occupancy, binary Bayesian filtering is used to update the probability. The fused, updated structured scene state is obtained and used for generating subsequent candidate decisions.

[0071] This application effectively reduces state transitions caused by single-frame sensor noise, occlusion, or accidental inference errors of large models by fusing the current structured scene state with the historical scene state, making environmental perception more continuous and stable, and providing a more reliable scene memory capability for fine maneuvers and long-term tasks.

[0072] In some optional implementations of this embodiment, the step of performing security constraint verification on candidate decisions to obtain valid decisions that pass the verification includes: Obtain safety constraints, which include at least one of the following: maximum speed, maximum acceleration, flight altitude range, geofence, minimum safe distance from obstacles, remaining battery threshold, location reliability threshold, and communication quality threshold. Candidate decisions are validated based on safety constraints, and those that pass the validation are considered valid decisions.

[0073] Safety constraints refer to the preset motion boundaries and state threshold conditions to ensure flight safety. Maximum speed constraints are the highest speed limits allowed for flight. Maximum acceleration constraints are the highest acceleration limits allowed for flight. Flight altitude range constraints are the lowest and highest altitude limits allowed for flight. Geofence constraints are the boundaries of flyable areas defined by geofence data. Minimum safe distance constraints from obstacles are the minimum distance that must be maintained between the aircraft and obstacles. Remaining battery threshold constraints are the lowest battery level required to ensure a safe return. Positioning reliability threshold constraints are the lower limit of positioning accuracy required to perform a specific action. Communication quality threshold constraints are the lowest communication quality threshold required to maintain the control link.

[0074] In this embodiment, safety constraints are obtained through pre-stored configuration parameters, real-time updated geofence data, and online estimated battery models and positioning covariance. A step-by-step decision pipeline is used for verification: first, kinematic constraints are verified, including maximum speed, maximum acceleration, and flight altitude range; then, spatial constraints are verified, including geofences and minimum safe distances from obstacles; finally, system resource constraints are verified, including remaining battery power thresholds, positioning reliability thresholds, and communication quality thresholds. A candidate decision is marked as failing if it violates any constraint; only candidate decisions that pass all constraint verifications are considered valid.

[0075] This application establishes an independent safety constraint verification layer between the perception big model and flight control execution, verifying each candidate decision to ensure it meets various safety boundary conditions. This effectively prevents unsafe or unexecutable big model decisions from directly entering the flight control system, thus fundamentally solving the risks of collisions, boundary violations, and loss of control caused by unreliable big model outputs.

[0076] In some optional implementations of this embodiment, the step of verifying candidate decisions based on security constraints and taking the verified candidate decisions as valid decisions includes: When there are candidate decisions that do not meet the safety constraints, the candidate decisions that do not meet the constraints are rejected, the speed is reduced, the obstacle avoidance distance is increased, or the plan is redesigned.

[0077] Rejection means directly discarding the candidate decision. Reducing speed means proportionally reducing the speed command in the candidate decision to below the maximum allowable value. Increasing obstacle avoidance distance means increasing the safety boundary around the obstacle and then replanning the trajectory. Replanning means regenerating candidate decisions using the revised constraints as input.

[0078] In this embodiment, when a candidate decision does not meet the safety constraints, a differentiated correction strategy is adopted according to the type and degree of violation: if it is only a slight speeding, the speed command in the decision is reduced proportionally to below the maximum speed constraint and re-verified; if the distance to a static obstacle is insufficient, the obstacle avoidance distance is expanded with the obstacle as the center, and a local path planning algorithm is called to generate a new trajectory that meets the safety distance nearby; if the candidate decision causes the remaining battery power to fall below the threshold or the communication quality to drop below the critical level, the candidate decision is directly rejected, and conservative solutions such as returning to home or landing are given priority.

[0079] This application, by taking differentiated measures such as rejection, speed reduction, increased obstacle avoidance distance, or replanning for candidate decisions that do not meet the constraints, rather than simply rejecting them all, can maximize the ability to generate effective decisions while ensuring flight safety, thus balancing mission success rate and flight safety.

[0080] In some optional implementations of this embodiment, the step of updating the structured scene state based on flight feedback data includes: The deviation between the actual flight results and the predicted results of the candidate decisions is determined based on flight feedback data; When the deviation exceeds the preset threshold, the steps of acquiring mission instructions, environmental perception data, and flight status data are re-executed until the structured scene status is updated based on the flight feedback data.

[0081] Deviation refers to the position or velocity error between the actual flight trajectory and the candidate decision prediction trajectory. The preset threshold is the deviation threshold value that triggers replanning.

[0082] In this embodiment, the flight feedback data is output by the navigation filter and includes estimates of the actual position, velocity, and attitude. By calculating the cumulative position error between the actual flight trajectory and the candidate decision prediction trajectory, when the deviation exceeds a preset threshold, the current execution process is immediately interrupted and the complete closed loop from acquiring mission instructions, environmental perception data, and flight status data to updating the structured scene state based on the flight feedback data is re-executed, generating a new decision that conforms to the latest environmental state.

[0083] This application actively monitors the deviation between the actual flight results and the predicted results and triggers a complete replanning closed loop when the deviation exceeds the limit. This enables the system to respond quickly to unexpected situations such as sudden obstacles, airflow disturbances or actuator delays, avoiding mission failure or dangerous approach due to accumulated errors. It significantly improves the system's robustness and autonomous adaptability in dynamic and complex environments.

[0084] In some optional implementations of this embodiment, the steps following the unification of multimodal features and task instruction input perception model to obtain structured scene state include: Monitor the inference time, output confidence, continuous output consistency, communication status, and sensor status of the large-scale perception model; When the inference time exceeds the preset time threshold, the output confidence is lower than the preset confidence threshold, the continuous outputs conflict with each other, the communication is interrupted, or the key sensor is abnormal, the newly generated candidate decisions of the perception model are stopped from being used to generate effective decisions, and the system switches to the local lightweight model, preset rule control, hovering, return, landing, or manual remote control mode.

[0085] Inference time refers to the time consumed by the perception model from input to output. Output confidence refers to the quantitative assessment of the reliability of the perception model's output. Continuous output consistency refers to the degree of change in the category and position of the main objects in the structured scene state across multiple adjacent frames. Degradation switching refers to stopping the use of the perception model's output and switching to a backup control mode when the perception model malfunctions. Local lightweight model refers to a computationally less demanding alternative model deployed on the onboard computing unit. Preset rule control refers to a control method based on fixed rules. Manual remote control mode refers to the mode in which ground operators directly control the drone via a remote control link.

[0086] In this embodiment, while the perception model performs inference, its inference time, output confidence score, consistency coefficient of consecutive multi-frame output results, link status of the airborne communication module, and data update flags and self-test status of each key sensor are monitored in parallel. When any of the following conditions are detected: inference time exceeds a preset time threshold, output confidence score is lower than a preset confidence threshold, consecutive outputs conflict, communication is interrupted, or a key sensor malfunctions, degradation protection is triggered. The newly generated candidate decisions of the perception model are stopped from being used to generate effective decisions, and the system switches to one of the following alternative control methods: local lightweight model, preset rule control, hovering, return to home, landing, or manual remote control.

[0087] This application ensures that the system can maintain a predictable and safe state when the core intelligent unit fails by continuously monitoring its operational status after the perception model has made inference and triggering a degradation switch when an anomaly occurs. This prevents loss of control accidents caused by model collapse or sudden environmental changes and provides the last line of defense for the entire autonomous flight system.

[0088] In some optional implementations of this embodiment, the step of generating candidate decisions based on the structured scene state, the mission objective corresponding to the mission instruction, and flight state data includes: Determine the current task phase; Based on the structured scenario state and flight state data, generate at least one candidate action or candidate trajectory corresponding to the current mission stage and mission objective. The candidate decision includes the candidate action or candidate trajectory and the corresponding decision confidence level. The candidate decisions are evaluated based on task completion, collision risk, energy consumption, trajectory smoothness, and communication status. Based on the evaluation results, the candidate decisions used for safety constraint verification are selected from the candidate decisions.

[0089] Mission phases refer to the various stages of a flight mission divided by time or space. Candidate maneuvers refer to the available basic flight maneuvers, including forward movement, circling, hovering, tracking, approaching, moving away, returning to base, and landing. Candidate trajectories refer to the spatial path and velocity profile from the current position to the target position. Decision confidence is a quantitative assessment of the reliability of a candidate decision in successfully completing the mission and ensuring safety. Mission completion rate refers to the estimated improvement in mission progress after executing a candidate decision. Collision risk refers to the distance between the candidate trajectory and obstacles, and the threat level of dynamic obstacles. Energy consumption refers to the electrical power consumed in executing a candidate trajectory. Trajectory smoothness refers to the rate of change of acceleration of the trajectory. Communication status refers to the signal strength with the ground station.

[0090] In this embodiment, the current task stage is determined by parsing task instructions and matching feature markers in the structured scene state. For example, when a target object is detected and the distance and viewpoint are confirmed to meet the shooting conditions, the task execution stage begins. Candidate trajectories are generated using a sampling-based motion planning algorithm (Rapidly-exploring Random Tree, RRT) or trajectory optimization methods to generate multiple smooth trajectories that satisfy dynamic constraints in the state space. Each trajectory corresponds to a candidate decision. Simultaneously, the perception model outputs action suggestions and confidence levels based on its understanding. The evaluation stage uses a weighted summation of five indicators: task completion, collision risk, energy consumption, trajectory smoothness, and communication status to obtain a comprehensive score. Based on this comprehensive score, the candidate decision with the highest score and not lower than a threshold is selected for safety constraint verification.

[0091] This application selects candidate decisions through mission phase determination and multi-indicator comprehensive evaluation, so that the candidate decisions entering the safety verification stage have achieved a better balance between mission achievement and flight safety, reducing the ineffective use of verification resources by low-quality candidates and improving the overall decision-making efficiency and execution quality.

[0092] The embodiments of this application can acquire and process relevant data based on artificial intelligence technology. Artificial intelligence (AI) refers to the theories, methods, technologies, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, perceive the environment, acquire knowledge, and use that knowledge to obtain optimal results.

[0093] Foundational technologies for artificial intelligence generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing, operating / interactive systems, and mechatronics. AI software technologies mainly encompass computer vision, robotics, biometrics, speech processing, natural language processing, and machine learning / deep learning.

[0094] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by instructing related hardware through computer-readable instructions. These computer-readable instructions can be stored in a computer-readable storage medium. When the program is executed, it can include the processes of the embodiments of the above methods. The aforementioned storage medium can be a non-volatile storage medium such as a magnetic disk, optical disk, or read-only memory (ROM), or random access memory (RAM).

[0095] It should be understood that although the steps in the flowcharts of the accompanying figures are shown sequentially as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the accompanying figures may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily completed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the sub-steps or stages of other steps.

[0096] Further reference Figure 3 As a response to the above Figure 2 The implementation of the method shown in this application provides an embodiment of an unmanned aerial vehicle (UAV) control system based on a perception model for autonomous perception and decision-making. This device embodiment is similar to... Figure 2 Corresponding to the method embodiments shown, this device can be specifically applied to various electronic devices.

[0097] like Figure 3 As shown, the UAV control system 300 based on a perception model for autonomous perception and decision-making described in this embodiment includes: a data acquisition module 301, a feature generation module 302, a scene generation module 303, a decision generation module 304, a security verification module 305, and an execution feedback module 306. Wherein: The data acquisition module is used to acquire mission instructions, environmental perception data, and flight status data; The feature generation module is used to synchronize environmental perception data and flight status data in time and transform coordinates to obtain synchronized data, and to encode the synchronized data to generate unified multimodal features. The scene generation module is used to input unified multimodal features and task instructions into the perception big model to obtain a structured scene state; The decision generation module is used to generate candidate decisions based on the structured scenario state, the mission objectives corresponding to the mission instructions, and flight status data. The security verification module is used to perform security constraint verification on candidate decisions and obtain valid decisions that pass the verification. The execution feedback module is used to convert effective decisions into flight control commands, control flight according to flight control commands, and update the structured scenario state based on flight feedback data.

[0098] The UAV control system based on a perception model for autonomous perception and decision-making provided in this application acquires mission instructions, environmental perception data, and flight status data, providing a complete raw data foundation for subsequent unified perception and mission understanding.

[0099] By synchronizing environmental perception data and flight status data in time and transforming coordinates, synchronized data is obtained. The synchronized data is then feature-encoded to generate unified multimodal features. Multi-source heterogeneous sensor data is transformed into a unified feature representation with spatiotemporal source and credibility information, enabling the perception big model to perform cross-modal reasoning in a unified semantic space. This solves the problem in existing technologies where different sensor data are processed independently and it is difficult to form a unified understanding.

[0100] By inputting unified multimodal features and task instructions into the perception big model, a structured scene state is obtained; the system acquires structured semantic information including environmental objects, spatial relationships, task objectives and risk areas, providing a complete scene cognition foundation for decision generation.

[0101] By generating candidate decisions based on structured scenario states, mission objectives corresponding to mission instructions, and flight status data, the decision generation process takes into account mission requirements, environmental constraints, and flight status, thereby improving the relevance and feasibility of the decisions.

[0102] By performing safety constraint verification on candidate decisions, valid decisions that pass the verification are obtained; an independent safety barrier is established between the output of the perception model and the flight controller to prevent untrusted decisions from directly entering the flight control system, thus solving the problems of collision, boundary crossing and loss of control caused by large model output delay, insufficient confidence or inability to execute.

[0103] By converting effective decisions into flight control commands, controlling flight according to flight control commands, and updating the structured scenario state based on flight feedback data, a closed loop of perception, decision-making, verification, execution, and feedback is formed. This enables the system to correct execution deviations in real time and adapt to dynamic environmental changes, thereby improving the safety, stability, and mission execution capabilities of UAV autonomous flight.

[0104] In some optional implementations of this embodiment, the feature generation module S202 is further configured to: Denoising and time-aligning are performed on environmental perception data and flight status data; Based on the sensor installation parameters and the drone's motion status, coordinate transformation and motion compensation are performed on the time-aligned data to obtain synchronized data. Extract the features corresponding to each data type in the synchronized data; The extracted features are appended with time identifiers, spatial location identifiers, sensor source identifiers, and confidence identifiers to obtain unified multimodal features.

[0105] The UAV control system based on the perception model provided in this application unifies the encoding of multi-sensor data after denoising, time alignment, coordinate transformation and motion compensation, and explicitly adds spatiotemporal source and credibility labels, so that sensor data with different physical meanings have a clear reference benchmark in a unified semantic space, eliminating perception ambiguity caused by data misalignment and improving the accuracy and robustness of subsequent environmental understanding.

[0106] In some optional implementations of this embodiment, the scene generation module S203 is further configured to: By using a large perception model to jointly reason about unified multimodal features and mission instructions, the system determines the category, location, speed, direction of motion of environmental objects and their spatial relationship with the UAV, and identifies passable areas, obstacle areas, mission objectives and risk areas. Transform the reasoning results into a structured scene state in the form of a scene graph, semantic map, object list, or state vector.

[0107] The UAV control system based on a perception model for autonomous perception and decision-making provided in this application uses a large perception model to jointly reason about multimodal features and task commands, enabling the system not only to detect objects in the environment, but also to understand the relationship between each object and the current task. This achieves a leap from fragmented perception to unified understanding at the task level, providing a complete semantic foundation for subsequent decision-making.

[0108] In some optional implementations of this embodiment, the scene generation module S203 is further configured to: Obtain historical scene status; The structured scene state output by the perception model is fused with the historical scene state to update the structured scene state.

[0109] The UAV control system based on the perception model for autonomous perception and decision-making provided in this application effectively reduces state jumps caused by single-frame sensor noise, occlusion, or accidental inference errors of large models by fusing the current structured scene state with the historical scene state, making the environmental perception more continuous and stable, and providing a more reliable scene memory capability for fine maneuvering and long-term missions.

[0110] In some optional implementations of this embodiment, the security verification module S205 is further configured to: Obtain safety constraints, which include at least one of the following: maximum speed, maximum acceleration, flight altitude range, geofence, minimum safe distance from obstacles, remaining battery threshold, location reliability threshold, and communication quality threshold. Candidate decisions are validated based on safety constraints, and those that pass the validation are considered valid decisions.

[0111] The UAV control system based on autonomous perception and decision-making provided in this application verifies whether candidate decisions meet various safety boundary conditions by setting an independent safety constraint verification layer between the perception model and flight control execution. This effectively prevents unsafe or unexecutable large model decisions from directly entering the flight control system, thus fundamentally solving the risks of collision, boundary crossing and loss of control caused by unreliable output of the large model.

[0112] In some optional implementations of this embodiment, the security verification module S205 is further configured to: When there are candidate decisions that do not meet the safety constraints, the candidate decisions that do not meet the constraints are rejected, the speed is reduced, the obstacle avoidance distance is increased, or the plan is redesigned.

[0113] The UAV control system based on the perception model provided in this application can retain the ability to generate effective decisions to the greatest extent while ensuring flight safety, by taking differentiated processing measures such as rejection, speed reduction, increasing obstacle avoidance distance or replanning for candidate decisions that do not meet the constraints, rather than simply rejecting them all. This balances the success rate of mission execution and flight safety.

[0114] In some optional implementations of this embodiment, the execution feedback module S206 is further configured to: The deviation between the actual flight results and the predicted results of the candidate decisions is determined based on flight feedback data; When the deviation exceeds the preset threshold, the steps of acquiring mission instructions, environmental perception data, and flight status data are re-executed until the structured scene status is updated based on the flight feedback data.

[0115] The UAV control system based on the perception model provided in this application actively monitors the deviation between the actual flight results and the predicted results and triggers a complete replanning closed loop when the deviation exceeds the limit. This enables the system to respond quickly to unexpected situations such as sudden obstacles, airflow disturbances or actuator delays, and avoids mission failure or dangerous approach due to accumulated errors. It significantly improves the robustness and autonomous adaptability of the system in dynamic and complex environments.

[0116] In some optional implementations of this embodiment, the scene generation module S203 is further configured to: Monitor the inference time, output confidence, continuous output consistency, communication status, and sensor status of the large-scale perception model; When the inference time exceeds the preset time threshold, the output confidence is lower than the preset confidence threshold, the continuous outputs conflict with each other, the communication is interrupted, or the key sensor is abnormal, the newly generated candidate decisions of the perception model are stopped from being used to generate effective decisions, and the system switches to the local lightweight model, preset rule control, hovering, return, landing, or manual remote control mode.

[0117] The UAV control system based on the perception model for autonomous perception and decision-making provided in this application continuously monitors the operating status of the perception model after inference and triggers a degradation switch when an anomaly occurs. This ensures that the system can still maintain a predictable and safe state when the core intelligent unit fails, preventing loss of control accidents caused by model collapse or sudden environmental changes, and providing the last line of defense for the entire autonomous flight system.

[0118] In some optional implementations of this embodiment, the decision generation module S204 is further configured to: Determine the current task phase; Based on the structured scenario state and flight state data, generate at least one candidate action or candidate trajectory corresponding to the current mission stage and mission objective. The candidate decision includes the candidate action or candidate trajectory and the corresponding decision confidence level. The candidate decisions are evaluated based on task completion, collision risk, energy consumption, trajectory smoothness, and communication status. Based on the evaluation results, the candidate decisions used for safety constraint verification are selected from the candidate decisions.

[0119] The UAV control system based on the perception model for autonomous perception and decision-making provided in this application selects candidate decisions through task phase judgment and multi-index comprehensive evaluation, so that the candidate decisions entering the safety verification stage have achieved a better balance between task achievement and flight safety, reducing the ineffective occupation of verification resources by low-quality candidates and improving the overall decision-making efficiency and execution quality.

[0120] To address the aforementioned technical problems, embodiments of this application also provide a computer device. Please refer to [link / reference needed]. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0121] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected via a system bus. It should be noted that only the computer device 4 with components 41, 42, and 43 is shown in the figure; however, it should be understood that it is not required to implement all the shown components, and more or fewer components can be implemented alternatively. Those skilled in the art will understand that the computer device described here is a device capable of automatically performing numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes, but is not limited to, microprocessors, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), digital signal processors (DSPs), embedded devices, etc.

[0122] The computer device can be a desktop computer, laptop, handheld computer, or cloud server, etc. The computer device can interact with the user via a keyboard, mouse, remote control, touchpad, or voice control.

[0123] The memory 41 includes at least one type of readable storage medium, including flash memory, hard disk, multimedia card, card-type memory (e.g., SD or DX memory), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, magnetic disk, optical disk, etc. In some embodiments, the memory 41 may be an internal storage unit of the computer device 4, such as the hard disk or memory of the computer device 4. In other embodiments, the memory 41 may also be an external storage device of the computer device 4, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc., equipped on the computer device 4. Of course, the memory 41 may include both the internal storage unit and its external storage device of the computer device 4. In this embodiment, the memory 41 is typically used to store the operating system and various application software installed on the computer device 4, such as computer-readable instructions for a drone control method based on a perception model for autonomous perception and decision-making. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or will be output.

[0124] In some embodiments, the processor 42 may be a central processing unit (CPU), controller, microcontroller, microprocessor, or other data processing chip. The processor 42 is typically used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to execute computer-readable instructions stored in the memory 41 or to process data, for example, to execute computer-readable instructions of the UAV control method based on the perception model's autonomous perception and decision-making.

[0125] The network interface 43 may include a wireless network interface or a wired network interface, which is typically used to establish communication connections between the computer device 4 and other electronic devices.

[0126] This application also provides another embodiment, namely, providing a computer-readable storage medium storing computer-readable instructions that can be executed by at least one processor to cause the at least one processor to perform the steps of the above-described UAV control method based on a perception model for autonomous perception and decision-making.

[0127] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the UAV control method based on perception model autonomous perception and decision-making described in the various embodiments of this application.

[0128] Obviously, the embodiments described above are only some embodiments of this application, not all embodiments. The accompanying drawings show preferred embodiments of this application, but do not limit the patent scope of this application. This application can be implemented in many different forms; rather, these embodiments are provided to provide a more thorough and comprehensive understanding of the disclosure of this application. Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing specific embodiments, or make equivalent substitutions for some of the technical features. Any equivalent structures made using the content of this application's specification and drawings, directly or indirectly applied to other related technical fields, are similarly within the scope of patent protection of this application.

Claims

1. A method for controlling unmanned aerial vehicles (UAVs) based on autonomous perception and decision-making using a perception model, characterized in that, Includes the following steps: Acquire mission instructions, environmental awareness data, and flight status data; The environmental perception data and the flight status data are synchronized in time and transformed in coordinates to obtain synchronized data, and the synchronized data is feature-encoded to generate unified multimodal features; The unified multimodal features and the task instructions are input into the perception big model to obtain the structured scene state; Based on the structured scenario state, the mission objective corresponding to the mission instruction, and the flight status data, candidate decisions are generated; The candidate decisions are subjected to security constraint verification to obtain valid decisions that pass the verification. The effective decision is converted into flight control commands, the flight is controlled according to the flight control commands, and the structured scene state is updated according to the flight feedback data.

2. The UAV control method based on autonomous perception and decision-making using a perception model according to claim 1, characterized in that, The process involves synchronizing and transforming the environmental perception data and the flight status data in time to obtain synchronized data, and then encoding the synchronized data to generate unified multimodal features, including: The environmental perception data and the flight status data are denoised and time-aligned. Based on the sensor installation parameters and the drone's motion state, coordinate transformation and motion compensation are performed on the time-aligned data to obtain the synchronized data. Extract the features corresponding to each data type in the synchronized data; The extracted features are further labeled with time identifiers, spatial location identifiers, sensor source identifiers, and confidence identifiers to obtain the unified multimodal features.

3. The UAV control method based on autonomous perception and decision-making using a perception model according to claim 1, characterized in that, The step of inputting the unified multimodal features and the task instructions into the perception big model to obtain the structured scene state includes: By jointly reasoning the unified multimodal features and the task instructions through the perception big model, the category, location, speed, direction of movement of environmental objects and their spatial relationship with the UAV are determined, and passable areas, obstacle areas, task objectives and risk areas are identified. The reasoning results are converted into the structured scene state in the form of a scene graph, semantic map, object list, or state vector.

4. The UAV control method based on autonomous perception and decision-making using a perception model according to claim 1, characterized in that, The step of inputting the unified multimodal features and the task instructions into the perception big model to obtain the structured scene state includes: Obtain historical scene status; The structured scene state output by the perception model is fused with the historical scene state to update the structured scene state.

5. The UAV control method based on autonomous perception and decision-making using a perception model according to claim 1, characterized in that, The step of performing security constraint verification on the candidate decisions to obtain valid decisions that pass the verification includes: Obtain safety constraints, which include at least one of the following: maximum speed, maximum acceleration, flight altitude range, geofence, minimum safe distance from obstacles, remaining battery threshold, location reliability threshold, and communication quality threshold. The candidate decisions are verified according to the security constraints, and the candidate decisions that pass the verification are taken as the valid decisions.

6. The UAV control method based on autonomous perception and decision-making according to claim 5, characterized in that, The step of verifying the candidate decisions according to the security constraints and taking the candidate decisions that pass the verification as the valid decisions includes: When there are candidate decisions that do not meet the aforementioned safety constraints, the candidate decisions that do not meet the constraints are rejected, their speed is reduced, their obstacle avoidance distance is increased, or they are replanned.

7. The UAV control method based on autonomous perception and decision-making using a perception model according to claim 1, characterized in that, The step of updating the structured scene state based on flight feedback data includes: The deviation between the actual flight results and the predicted results of the candidate decisions is determined based on the flight feedback data. When the deviation exceeds a preset threshold, the step of acquiring the task instruction, environmental perception data, and flight status data is re-executed until the structured scene status is updated based on the flight feedback data.

8. The UAV control method based on autonomous perception and decision-making using a perception model according to claim 1, characterized in that, After inputting the unified multimodal features and the task instructions into the perception big model to obtain the structured scene state, the process includes: Monitor the inference time, output confidence, continuous output consistency, communication status, and sensor status of the perception model; When it is detected that the inference time exceeds a preset time threshold, the output confidence is lower than a preset confidence threshold, continuous outputs conflict with each other, communication is interrupted, or key sensors are abnormal, the candidate decisions newly generated by the perception model are stopped from being used to generate the effective decision, and the system switches to the local lightweight model, preset rule control, hovering, return, landing, or manual remote control mode.

9. The UAV control method based on autonomous perception and decision-making using a perception model according to claim 1, characterized in that, The step of generating candidate decisions based on the structured scenario state, the task objective corresponding to the task instruction, and the flight state data includes: Determine the current task phase; Based on the structured scene state and the flight state data, at least one candidate action or candidate trajectory is generated corresponding to the current mission stage and the mission objective. The candidate decision includes the candidate action or candidate trajectory and the corresponding decision confidence level. The candidate decisions are evaluated based on task completion, collision risk, energy consumption, trajectory smoothness, and communication status, and the candidate decisions used for safety constraint verification are determined from the candidate decisions based on the evaluation results.

10. A UAV control system based on a perception model for autonomous perception and decision-making, characterized in that, include: The data acquisition module is used to acquire mission instructions, environmental perception data, and flight status data; The feature generation module is used to synchronize the environmental perception data and the flight status data in time and transform them in coordinates to obtain synchronized data, and to encode the synchronized data to generate unified multimodal features. The scene generation module is used to input the unified multimodal features and the task instructions into the perception big model to obtain the structured scene state; The decision generation module is used to generate candidate decisions based on the structured scenario state, the task objective corresponding to the task instruction, and the flight status data; The security verification module is used to perform security constraint verification on the candidate decisions to obtain valid decisions that pass the verification. The execution feedback module is used to convert the effective decision into flight control commands, control the flight according to the flight control commands, and update the structured scene state according to the flight feedback data.

Images