Method for expanding unmanned aerial vehicle navigation during testing based on semantic and physical perception

Abstract

The invention discloses a semantic and physical perception-based unmanned aerial vehicle navigation extension method during testing. The method comprises the following steps: S1, constructing multi-modal navigation input information of an unmanned aerial vehicle; s2, based on multi-modal navigation input information, judging whether a unique candidate navigation task target conforming to the description of the navigation task instruction exists in the current view image or not, and correspondingly constructing an initial candidate waypoint set; s3, generating corrected candidate waypoints and a corrected candidate waypoint set by using the visual language model; s4, performing multi-dimensional quantitative scoring on the corrected candidate waypoints in the corrected candidate waypoint set; and S5, selecting the corrected candidate waypoint with the highest multi-dimensional quantitative score as an optimal waypoint, and generating a track containing a continuous pose sequence based on the optimal waypoint to control the unmanned aerial vehicle. According to the method, the navigation planning and self-correction capability can be enhanced, and the navigation decision accuracy and reliability are effectively improved.

Description

Technical Field

[0001] This invention relates to the field of autonomous navigation technology for unmanned aerial vehicles (UAVs), and in particular to a test-time extended visual language navigation method for UAVs based on active semantic verification and physical perception. Background Technology

[0002] In recent years, unmanned aerial vehicle (UAV) technology has developed rapidly, and its applications have penetrated into many key areas such as automated logistics, large-scale infrastructure inspection, precision agriculture, and even disaster search and rescue. For UAVs to perform tasks efficiently and safely in these complex scenarios, their autonomous navigation capability is a core technological bottleneck. UAV navigation research has long focused on two core challenges: first, maintaining accurate self-positioning in environments where Global Navigation Satellite System (GNSS) signals are limited or absent; and second, ensuring flight path safety in complex scenarios filled with dynamic obstacles.

[0003] To address these challenges, early research focused primarily on multi-sensor fusion techniques. For example, visual inertial odometry (VIO), which fuses visual and inertial measurement unit (IMU) information, combined with LiDAR or ultra-wideband (UWB) technology, can achieve robust relative positioning to some extent. However, while these low-level sensor fusion-based methods solve the "where am I?" problem, they struggle with complex, long-distance navigation tasks requiring advanced semantic understanding. For instance, they cannot understand semantically rich human commands such as "fly to that red-roofed building."

[0004] To bridge the gap between machine perception and human intent, cutting-edge research has begun to introduce Large Language Models (LLMs) and Visual Language Models (VLMs), which possess powerful cognitive capabilities, into the decision-making process of UAV navigation. VLMs can simultaneously process and understand images and natural language, making it possible to establish a more intuitive and flexible human-computer interaction paradigm. With the help of VLMs, non-professional users can control UAVs to perform complex tasks using only simple natural language commands, greatly lowering the operational threshold for UAVs and driving the rapid development of the field of visual language navigation research.

[0005] In the field of Visual Language Navigation (VLN), pioneering works such as AerialVLN and UAV-Need-Help have successfully validated the basic feasibility of VLMs converting language commands into flight trajectories by providing realistic simulation environments and benchmark datasets. Building upon this foundation, research in this field has continued to deepen, giving rise to several technical optimization directions: some studies focus on building end-to-end language-guided flight systems that are closer to real-world applications; others focus on more refined navigation command grounding mechanisms, addressing performance bottlenecks caused by inaccurate alignment between commands and visual information in early datasets by introducing methods such as entity-landmark alignment; still others are exploring how to enhance the model's long-term decision-making capabilities, such as using thought chain reasoning to assist VLMs in achieving better global path planning. While these studies have made some progress by improving model architecture or optimizing training strategies, their underlying decision-making models mostly still haven't broken free from the framework of single-step decision-making. Upon receiving the current multimodal input, the model directly generates an action command, and its decision-making process resembles a computationally efficient instantaneous reaction. This approach works well in simple scenarios, but its reliability drops significantly in complex environments requiring careful consideration. Therefore, improving the decision-making quality and robustness of VLM navigation models has become a key challenge in the intersection of artificial intelligence and robotics.

[0006] Semantic verification is a key technology aimed at evaluating and confirming the consistency between machine perception results and pre-defined semantic targets. It is widely used in computer vision, visual language navigation, human-computer interaction, and embodied intelligence. The core mechanism of this method lies in constructing a discriminative model. By calculating the degree of matching between visual observation data (such as images, video frames, or point clouds) and high-level semantic descriptions (such as natural language instructions or category labels) in the feature space, it reconfirms whether the initially detected object is a real target entity, thereby filtering out false positives caused by environmental noise, texture similarity, or cluttered backgrounds. Early semantic verification techniques relied heavily on hard-coded geometric rule matching or confidence threshold determination based on single-modality classifiers, resulting in limited generalization ability. With the evolution of deep learning technology, current mainstream research focuses on cross-modal alignment methods based on large-scale visual language models (such as CLIP and BLIP). These methods typically map image features and text features to a unified high-dimensional embedding space, measuring their semantic relevance by calculating cosine similarity or attention weights. Applying semantic verification methods to UAV visual language navigation can reduce the uncertainty of visual perception in complex and unstructured environments, prevent "semantic illusions" caused by one-way feature matching, and improve the accuracy of the system when executing instructions.

[0007] Physical perception refers to the techniques used by intelligent systems to reconstruct the three-dimensional geometry of the environment and determine the precise location of targets in physical space using sensor data. In the fields of visual navigation and robotics, physical perception not only requires the system to identify what the target is, but also to accurately calculate where the target is, that is, to realize the transformation from pixel coordinates in a two-dimensional image plane to a three-dimensional world coordinate system. In existing technologies, physical perception is usually achieved by relying on depth cameras (RGB-D), LiDAR, or depth estimation algorithms based on binocular / monocular vision. The system acquires the depth information of the scene and combines it with camera intrinsic parameters to backproject image pixels into a three-dimensional point cloud, thereby constructing a local or global map of the environment. Applying physical perception methods to UAV visual language navigation can accurately map abstract image understanding into three-dimensional spatial coordinates in the real world, thereby giving intelligent agents the ability to perform reliable navigation and precise interaction in complex environments.

[0008] Test-time scaling is an emerging paradigm for improving model performance during the inference phase. Its core lies in dynamically increasing computational resources allocated during inference, without modifying model parameters or retraining, to significantly improve the model's decision-making quality when handling complex tasks. The basic principle is to increase the computational budget during inference to unlock and enhance the model's inherent, but underutilized, deep reasoning capabilities. This concept has been validated in multiple technical fields, including dynamic inference and test-time adaptation, demonstrating that investing more computation during inference is an effective way to improve model performance. Therefore, applying test-time scaling to UAV visual-language navigation provides a novel technical approach to addressing the inherent limitations of existing single-step decision-making models. Summary of the Invention

[0009] The purpose of this invention is to provide a test-time extended UAV navigation method based on semantic and physical perception, which can enhance the planning and self-correction capabilities of navigation and effectively improve the accuracy and reliability of navigation decisions.

[0010] The present invention adopts the following technical solution:

[0011] A test-time extended UAV navigation method based on semantic and physical perception includes the following steps:

[0012] S1: The UAV acquires image and depth data and combines it with the navigation mission objectives in the received navigation mission instructions to construct the UAV's multimodal navigation input information;

[0013] S2: Based on the multimodal navigation input information, determine whether there exists a unique candidate navigation task target within the current field of view that matches the navigation task instruction description:

[0014] If so, determine the three-dimensional position of the unique candidate navigation mission target and construct an initial set of candidate waypoints containing only a single, certain navigation mission target;

[0015] If not, identify multiple potential associated landmark areas in the current field of view that are related to the navigation task objective, and generate several initial candidate waypoints in parallel based on the center of each potential associated landmark area, and construct an initial candidate waypoint set containing multiple initial candidate waypoints;

[0016] S3: Based on the initial set of candidate waypoints and the set self-reflective prompts, obtain the prompts after injecting the candidate waypoints; then perform physical safety checks on the candidate waypoints to generate feedback information; subsequently, send the prompts after injecting the candidate waypoints, the feedback information, and the multimodal navigation input information back into the visual language model to generate corrected candidate waypoints and a set of corrected candidate waypoints after serial reflection optimization.

[0017] S4: Perform multi-dimensional quantitative scoring on the corrected candidate waypoints in the corrected candidate waypoint set;

[0018] S5: Select the corrected candidate waypoint with the highest multi-dimensional quantization score as the optimal waypoint, and use the optimal waypoint to generate a trajectory control UAV containing a continuous pose sequence.

[0019] Step S2 specifically includes the following steps:

[0020] S21: Using open vocabulary target detection technology, the natural language instructions and real-time RGB image observation set are correlated and inferred to determine whether there are candidate navigation task targets in the current field of view that match the description of the navigation task instructions.

[0021] If one or more candidate navigation task targets are detected in the current field of view, then enter the target locking mode and execute step S22; if no candidate navigation task targets are detected in the current field of view, then enter the semantic search mode and execute step S23.

[0022] S22: Perform semantic matching judgment on each candidate navigation task target; if the semantics do not match, the candidate navigation task target is determined to be a false detection and is removed; if the semantics match, the current candidate navigation task target is retained as the real target.

[0023] Once all candidate navigation task targets within the field of view have been matched, determine whether only one true target remains.

[0024] If there is only one real target, determine the three-dimensional position of the real target and then proceed to step S24; if there is no real target or there are multiple real targets, proceed to step S23.

[0025] S23: Using semantic association reasoning, identify multiple potential associated landmark regions that are relevant to the navigation task target from the current real-time RGB image observation set, and calculate the center of each associated landmark region to generate multiple corresponding search guidance points; then select several preferred search guidance points based on semantic relevance.

[0026] S24: If in target locking mode, the mission endpoint coordinates are directly encapsulated into an initial candidate waypoint set containing only a single element;

[0027] If in semantic search mode, then combining real-time image observation sets and navigation task objectives, and utilizing a visual language model, in Within the semantic constraint region of each preferred search guide point, parallel generation is performed on each preferred search guide point. 1 initial candidate waypoint, construct an initial candidate waypoint set.

[0028] Step S3 specifically includes the following steps:

[0029] S31: For each candidate waypoint in the initial candidate waypoint set, perform a... The iterative self-correcting process of the wheel;

[0030] S32: In each round of self-correction, the coordinate values ​​of the candidate waypoints are used to replace the waypoint coordinate correction placeholders in the self-reflective prompts, resulting in the self-reflective prompts after injecting the candidate waypoints;

[0031] S33: For each candidate waypoint, detect the distance between the drone and the surrounding environment, and generate additional natural language feedback based on the detection results;

[0032] S34: Combine the original multimodal navigation input information corresponding to the candidate waypoint, the self-reflective prompts and safety feedback information after injecting the candidate waypoint, and send them back to the visual language model for processing to finally generate the corrected candidate waypoints after reflection and optimization.

[0033] S35: Perform the above method on all candidate waypoints. The iterative correction round gathers all the corrected candidate waypoints that have been reflected upon and optimized into a corrected candidate waypoint set.

[0034] Step S4 specifically includes the following steps:

[0035] S41: Construct a multi-dimensional scoring function formed by weighted summation of spatiotemporal probabilistic risk, VLM attention evidence, and dynamic energy consumption ratio;

[0036] S42: Construct a spatiotemporal probabilistic risk scoring function;

[0037] S43: Construct the VLM attention evidence scoring function;

[0038] S44: Construct a dynamic energy consumption ratio scoring function;

[0039] S45: Calculate the multi-dimensional quantitative score for each corrected candidate waypoint using a multi-dimensional scoring function.

[0040] In step S22: cross-modal challenge is performed on the image region, and an image description of the region is generated in reverse and semantically compared with the navigation task instruction; if the semantics of the generated image description matches the navigation task instruction, the candidate navigation task target is determined to be consistent with the navigation task instruction description and confirmed as a real target; if the semantics of the generated image description does not match the navigation task instruction, the candidate navigation task target is determined to be inconsistent with the navigation task instruction description and is a false detection.

[0041] In step S22: After confirming the unique real target, the depth data corresponding to the real target area in the real-time depth image set is extracted, and the real target is reconstructed into local three-dimensional point cloud data in real time by combining the camera intrinsic parameters; then the density clustering algorithm is used to calculate the three-dimensional centroid coordinates of the real target, and the three-dimensional centroid coordinates are projected to the nearest flightable altitude and recorded as the mission endpoint coordinates as a unique element for temporary storage.

[0042] In step S23: First, a visual language model is used to infer potential associated landmark areas related to the target; then, combined with an airborne semantic map, the locations of potential associated landmark areas in the current field of view are retrieved in three-dimensional space, and local three-dimensional point clouds are constructed respectively; then, a density clustering algorithm is used to calculate the three-dimensional centroid coordinates of the local three-dimensional point cloud; then, these three-dimensional centroid coordinates are projected onto the nearest flightable altitude to generate multiple corresponding search guidance points; finally, based on semantic relevance, the target is selected... A preferred search guide point.

[0043] The spatiotemporal probabilistic risk scoring function adopts an exponential decay function constructed based on the expected collision risk and a set risk aversion coefficient. The expected collision risk is constructed based on the degree of overlap risk between the UAV's probabilistic existence pipeline and the probability density field of obstacle existence in the spatiotemporal domain. The probabilistic existence pipeline is constructed based on the UAV's three-dimensional position coordinates at each time, the expected position vector of the nominal trajectory at the corresponding time, and the covariance matrix of the position estimation error at that time. The covariance matrix contains process noise accumulated over time. The probability density field of obstacle existence is constructed by superimposing the Gaussian kernel functions corresponding to each obstacle point in the local spatial point cloud. The nominal trajectory is a smooth spatiotemporal curve connecting the corrected candidate waypoint, the UAV's current position, and the target point.

[0044] The VLM attention evidence scoring function employs a hyperbolic tangent function constructed based on the total information gain of the key evidence set at the waypoint and a set gain sensitivity coefficient. The key evidence set is constructed based on the image block with the highest weight in the cross-modal attention graph generated by the visual language model and its corresponding 3D surface normal vector. The total information gain is constructed based on the combined effect of the geometric perspective gain of each evidence block in the key evidence set. The geometric perspective gain is constructed based on the relationship between the alignment of the observation direction at the future time and the observation direction at the current time with respect to the target surface normal.

[0045] The dynamic energy consumption ratio scoring function is the ratio of the effective progress based on the UAV's position to the total energy consumption cost functional of the UAV flying along the trajectory. The effective progress is constructed based on the effective propulsion component of the displacement direction of the candidate trajectory in the global target direction. The total energy consumption cost functional is constructed based on the cumulative instantaneous total power of the UAV over the entire time period. The instantaneous total power is constructed based on the induced power required to maintain hovering and lift, the parasitic power to overcome air resistance, and the control power generated by attitude adjustment. The induced power, parasitic power, and control power are determined by the rotor total thrust vector, the instantaneous velocity, and the instantaneous angular velocity vector of the UAV, respectively, calculated based on the inverse dynamics model.

[0046] Compared with the prior art, the present invention has the following advantages:

[0047] 1. No additional training or fine-tuning of the model is required, resulting in high cost-effectiveness.

[0048] Unlike traditional methods that rely on improving model architecture or retraining, this invention requires no parameter modification or fine-tuning of any existing visual language model. This means that the high GPU computing power, long time cycle, and massive data annotation costs required to train large models can be completely avoided. By guiding a "frozen" model to think more deeply during the inference phase to unlock its inherent potential, the technology significantly reduces the threshold and cost of application, providing extremely high cost-effectiveness and convenience for the rapid deployment and upgrading of existing UAV visual language navigation systems.

[0049] 2. Accurate semantic cognition and extremely high positioning.

[0050] Existing technologies often blindly trust the preliminary detection results of visual models, easily leading to "semantic illusions" (such as mistaking a red trash can for a fire hydrant), and the output is limited to two-dimensional image regions, lacking physical depth. This invention innovatively introduces an "active semantic verification" mechanism, which removes potential semantic interference terms before planning by cross-modal cross-questioning (allowing the model to describe object attributes in reverse and compare them with instructions). Simultaneously, a 3D density clustering algorithm combining depth information transforms ambiguous visual targets into precise three-dimensional physical coordinates, improving the success rate of tasks in complex semantic environments.

[0051] 3. Possesses keen physical perception, prioritizing both safety and energy efficiency.

[0052] Unlike traditional methods that rely solely on simple geometric distances for obstacle avoidance, this invention constructs a multi-dimensional evaluation system based on a physical perception field. Through spatiotemporal probability risk scoring, the drone is no longer treated as a point, but rather as a spatiotemporal probability pipeline, quantifying the positioning uncertainty risk caused by sensor errors and enabling predictive obstacle avoidance. Through VLM attention evidence scoring, the drone is endowed with proactive perception capabilities, guiding it to actively plan paths that allow for more accurate and clearer observation of key semantic evidence, thereby eliminating visual ambiguity and uncertainty. Simultaneously, the kinetic energy consumption ratio scoring introduces an inverse dynamics model, automatically selecting trajectories that conform to fluid dynamics smoothness while maximizing power saving. This design ensures that the paths generated by the drone are not only semantically correct but also physically safe and energy-efficient, effectively extending the drone's operational endurance and reducing the risk of accidents.

[0053] 4. The decision-making process is more rigorous and robust.

[0054] This invention enhances navigation robustness by elevating navigation decision-making from intuitive reactions to deliberate planning through a rigorous three-stage reasoning framework. First, in the parallel exploration phase, the model is forced to generate multiple parallel candidate solutions, broadening the decision-making perspective and avoiding the risk of prematurely falling into local optima due to chance. Then, in the serial reflection phase, each candidate solution undergoes iterative correction and optimization by the model using self-reflective prompts. This process deepens the model's thinking, enabling it to identify and correct potential flaws in the initial plan. This mechanism, integrating broad exploration and deep optimization, ensures that the final navigation command output is the result of multi-faceted comparison and deep optimization. Therefore, its reliability surpasses traditional single-step decision-making models when facing ambiguous commands or complex environments. Attached Figure Description

[0055] Figure 1 This is a schematic diagram of the process of the present invention. Detailed Implementation

[0056] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numerals in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this disclosure. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this disclosure as detailed in the appended claims.

[0057] To more clearly demonstrate the specific implementation method of this invention, some related concepts and theories are first described:

[0058] Vision-Language Navigation (VLN) refers to enabling intelligent agents like drones to understand natural language commands and perform movement and navigation tasks in the real physical world, much like humans do. It's not merely simple target recognition or path planning, but a deep integration of language understanding, visual perception, and motion control. For example, a drone must not only see a fire hydrant, but also understand the description of the hydrant's location and the surrounding environment in the commands, and plan a safe and efficient flight path accordingly.

[0059] Test-Time Scaling is a technique that improves model performance by dynamically adjusting computational resources during the inference phase. Its core idea is to unlock the model's inherent but underutilized deep reasoning capabilities by allocating more computational resources during the inference (i.e., "testing") phase, without changing any internal parameters of the Virtual Model (VLM) or retraining it. In drone navigation applications, this means allowing the existing model to perform more rounds of reasoning in the few seconds the drone needs to make a decision, rather than spending weeks training a larger model.

[0060] Active Semantic Verification (ASVV) is a machine perception strategy inspired by cognitive psychology, designed to address the illusion problem commonly found in deep learning models. Traditional perception is passive (believing whatever the model outputs), while ASVV introduces a closed-loop "hypothesis-verification" logic: it first accepts the model's initial detection results as hypotheses, then actively gathers evidence by generating reverse descriptions (such as Visual Question Answering, VQA) or multi-view observations, and logically compares this new evidence with the original definition. Only the target that passes the "cross-questioning" is considered true. This method upgrades perception from one-way feature matching to two-way logical reasoning.

[0061] Kernel density estimation (KDE) is a nonparametric statistical method for estimating the probability density function of a random variable. When dealing with discrete sensor data (such as sparse point clouds generated by LiDAR or depth cameras), traditional geometric methods often focus only on the specific point coordinates. KDE, however, smooths the discrete point set into a continuous, differentiable probability density field by placing a kernel function (usually a Gaussian kernel) at each data point and superimposing these kernel functions. In the field of robot navigation, this theory is often used to construct the risk field of the environment. It does not treat obstacles as deterministic geometric objects but rather as a region of probability distribution, thus better handling sensor measurement noise and localization uncertainties.

[0062] The Cross-Modal Cross-Attention Mechanism is a core algorithmic component in multimodal large-scale models for understanding the association between vision and language. Mathematically, it generates a weight matrix (i.e., an attention map) by calculating the dot product similarity between the query vector and the key / value vector. This mechanism can not only identify objects in an image but also precisely quantify which specific pixel region in the image is strongly associated with which word in the text. It enables the model to accurately pinpoint abstract linguistic symbols to specific visual features, forming the underlying mathematical foundation for accurate visual localization.

[0063] Inverse dynamics is a classic physical concept in robotics and control theory, referring to the process of working backward from known motion states (position, velocity, acceleration) to deduce the forces and torques required to produce that motion. In contrast to forward dynamics (which seeks motion given forces), inverse dynamics focuses on costs. In the field of unmanned aerial vehicles (UAVs), it is based on the Newton-Euler equations. By analyzing the kinematic parameters of the UAV on a specific trajectory and combining them with aerodynamic coefficients, it works backward to deduce the total thrust required by the rotor and the power needed to overcome air resistance. This theory serves as a crucial mathematical bridge between geometric path planning and physical-level energy consumption assessment and feasibility verification.

[0064] like Figure 1 As shown, the test-time extended UAV navigation method based on semantic and physical perception described in this invention includes the following steps:

[0065] S1: The UAV receives natural language instructions containing navigation task instructions, collects image and depth data, and constructs multimodal navigation input information for the UAV by combining the navigation task objectives in the navigation task instructions;

[0066] In step S1, after receiving the navigation task command from the user's natural language instructions, the UAV acquires multimodal navigation input information. ;

[0067] in, , representing the real-time RGB image observation set collected by k cameras with different perspectives (front, back, left, right, and bottom) mounted on a drone. This represents the real-time image captured by the camera at the k-th viewpoint in the real-time image observation set; This represents a real-time depth image set corresponding to the viewpoint of the real-time image observation set, used to provide distance information about the environment to construct a 3D point cloud. This represents the depth map captured by the camera at the k-th viewpoint in the real-time depth image set; Represents natural language instructions given by the user that describe the goal of the navigation task;

[0068] The natural language commands given by the user include navigation task commands and action commands for the drone; the navigation task commands include navigation task objectives.

[0069] Upon receiving the user's navigation task trigger command, the UAV's visual-language navigation system first enters data acquisition mode. Simultaneously, the system utilizes its onboard multi-view camera array to capture a set of real-time RGB image observations, including front, rear, left, right, and bottom views. It also acquires real-time depth image sets strictly corresponding to each viewpoint, constructing complete 3D visual observation data. At the same time, the UAV's visual-language navigation system also receives natural language commands from the user regarding the navigation task objective, such as "fly to the red fire hydrant in the corner of the parking lot." The aforementioned visual observation data and natural language commands (i.e., real-time image observation sets, real-time depth image sets, and text commands) are integrated into a single multimodal navigation input message. .

[0070] S2: Based on multimodal navigation input information, open-vocabulary target detection technology is used to identify candidate navigation task targets, and cross-modal active semantic verification is performed on the candidate navigation task targets to eliminate false detections, thereby determining whether there is a unique candidate navigation task target in the current field of view that matches the navigation task instruction description:

[0071] If so, determine the three-dimensional position of the unique candidate navigation objective and construct an initial set of candidate waypoints containing only the single, certain navigation objective. ;

[0072] If not (i.e., the current field of view does not contain or contains multiple candidate navigation task targets that match the navigation task instruction description), then semantic association reasoning is used to identify multiple potential associated landmark regions in the current field of view that are strongly correlated with the navigation task target (greater than a preset correlation threshold), and the center of each potential associated landmark region is calculated to generate multiple corresponding search guidance points; then, based on semantic relevance, several preferred search guidance points are selected, and for each preferred search guidance point, several initial candidate waypoints are generated in parallel within the semantic constraint region of the preferred search guidance point, constructing an initial candidate waypoint set containing multiple initial candidate waypoints. .

[0073] In this invention, step S2 specifically includes the following steps:

[0074] S21: The multimodal navigation input information is fed into the VLM-3D fusion adaptive hierarchical perception module. Using open vocabulary target detection technology, the natural language instructions are correlated and inferred with the real-time RGB image observation set to determine whether there are candidate navigation task targets in the current field of view that match the navigation task instruction description.

[0075] If one or more candidate navigation task targets are detected in the current field of view, the UAV visual language navigation system enters the target locking mode and proceeds to step S22; if no candidate navigation task targets are detected in the current field of view, the UAV visual language navigation system enters the semantic search mode and proceeds to step S23.

[0076] S22: Perform semantic matching judgment on each candidate navigation task target; if the semantics do not match, the candidate navigation task target is judged as a false detection (semantic illusion) and is removed; if the semantics match, the current candidate navigation task target is retained as the real target;

[0077] Once all candidate navigation task targets within the field of view have been matched, determine whether only one true target remains.

[0078] If there is only one real target, determine the three-dimensional position of the real target and then proceed to step S24;

[0079] If there is no real target or there are multiple real targets, proceed to step S23;

[0080] Open-vocabulary object detection is an advanced visual perception technique that breaks through the limitation of traditional detection models, which can only recognize predefined closed categories. It utilizes multimodal knowledge (such as visual-linguistic feature alignment) pre-trained on large-scale image-text pairs to establish a semantic mapping between image regions and natural language descriptions. This gives the model powerful zero-shot generalization capabilities, allowing it to accurately locate previously unseen target objects in complex scenes based on arbitrary text prompts input by the user, without retraining. Open-vocabulary object detection is a standard technique in this field and will not be elaborated upon further.

[0081] In this embodiment, firstly, open-vocabulary target detection technology is used to identify candidate navigation task targets in a real-time RGB image observation set, and the image region corresponding to the candidate navigation task target is extracted. Then, cross-modal challenge is performed on the image region, i.e., an image description of the region is generated in reverse and semantically compared with the navigation task command. If the generated image description matches the semantics of the navigation task command, the candidate navigation task target is determined to be consistent with the navigation task command description and confirmed as a real target. If the generated image description does not match the semantics of the navigation task command, the candidate navigation task target is determined to be inconsistent with the navigation task command description, and is thus considered a false detection (semantic illusion) and is removed.

[0082] After identifying the unique real target, the real target region is extracted from the real-time depth image set. The system uses the corresponding depth data and camera intrinsics to reconstruct the real target into local 3D point cloud data in real time. Then, a density clustering algorithm is used to calculate the 3D centroid coordinates of the real target, and these coordinates are projected onto the nearest flyable altitude and recorded as the mission endpoint coordinates. Finally, the coordinates of the mission endpoint. It is temporarily stored as the only element.

[0083] S23: Using semantic association reasoning, identify multiple potential associated landmark regions that are strongly correlated with the navigation task target from the current real-time RGB image observation set, and calculate the center of each associated landmark region to generate multiple corresponding search guidance points; then select several preferred search guidance points based on semantic relevance.

[0084] In this embodiment, the common sense reasoning ability of the visual language model is first used to analyze the instructions. The system infers potential related landmark areas with strong correlation to the target (e.g., inferring "sidewalk" from "fire hydrant" in the command); then, combining the airborne semantic map, it retrieves the locations of potential related landmark areas in the current field of view in three-dimensional space, constructing local three-dimensional point clouds for each; then, it uses a density clustering algorithm to calculate the three-dimensional centroid coordinates of these local three-dimensional point clouds, projects these three-dimensional centroid coordinates onto the nearest flightable altitude, generating multiple corresponding search guidance points; finally, it filters out based on semantic relevance. Select a preferred search guide point, and then proceed to step S24.

[0085] S24: If in target-locked mode (from S22), then directly set the mission endpoint coordinates. (i.e., the three-dimensional position of the real target) is encapsulated into an initial set of candidate waypoints containing only a single element. ;

[0086] If in semantic search mode (from S23), then combining the real-time image observation set and the navigation task objective, and utilizing a visual language model, in Within the semantic constraint region of each preferred search guide point, parallel generation is performed on each preferred search guide point. 1 initial candidate waypoint, construct an initial candidate waypoint set. .

[0087] In this embodiment, the acquired multimodal navigation input information can be... An adaptive hierarchical perception module based on VLM-3D fusion is fed in. Depending on the target's visibility within the current field of view, the UAV's visual-language navigation system automatically switches between "target locking mode" and "semantic search mode." In locking mode, point clouds are reconstructed using depth data and semantic verification, the coordinates of the mission endpoint are calculated, and an initial set of candidate waypoints is constructed as a single element. In search mode, semantic reasoning is used to determine the exploration area, and the visual language model is analyzed. Independent inference calls are generated in parallel. Coordinates of the initial candidate waypoints Finally, a set containing all initial candidate waypoints is constructed. .

[0088] A Vision-Language Model (VLM) is a multimodal artificial intelligence model capable of jointly processing image and text data. Its core principle is to learn cross-modal alignment relationships to achieve a comprehensive understanding and generation of visual and linguistic information. In UAV navigation tasks, VLMs can simultaneously process and understand two completely different information sources: real-time visual images captured by the UAV's camera and natural language commands input by human users. VLMs are a standard technique in this field and will not be elaborated upon further.

[0089] The VLM-3D fusion adaptive hierarchical perception module is a core intelligent decision-making component integrating visual language deep understanding and three-dimensional physical perception capabilities. It aims to process multimodal navigation inputs and output high-confidence initial waypoints. This module consists of three functional units: environmental state discrimination, target lock verification, and semantic association search. Its core function is to adaptively and dynamically switch between "target lock" and "semantic search" modes based on the association reasoning results of natural language commands and real-time RGB-D observation data. In target lock mode, a cross-modal cross-questioning mechanism is introduced. Semantic illusions are eliminated by reverse-generating visual image descriptions and comparing them with commands. After confirming a unique real target, the three-dimensional centroid of the real target is accurately calculated using depth information and density clustering algorithms, projected onto the nearest flyable altitude, and recorded as the mission endpoint coordinates. In semantic search mode (when no target is detected, all candidate targets are disproven, or multiple real targets exist that cannot be uniquely locked), common sense reasoning is used to identify strongly correlated landmark areas with the target. By selecting and optimizing search guidance points, diverse exploratory waypoints are generated in parallel.

[0090] In this invention, navigation task initialization and data acquisition can be achieved through the above steps. When a unique real target that matches the navigation task instruction description in the natural language command is detected, the UAV visual language navigation system transforms its position into a high-confidence set of single initial candidate waypoints. The waypoint undergoes physical safety checks and energy efficiency assessments in subsequent steps. If no single true target matching the navigation task instructions in the natural language command is detected (including missing targets or ambiguous target directions), a breadth-first parallel exploration strategy is retained to generate multiple potential initial candidate waypoints, providing a rich and diverse pool of initial options for subsequent decisions. Through this design, step S2 can generate a standardized set of initial candidate waypoints regardless of the mode. This provides a unified input basis for subsequent steps, ensuring that path planning has rigorous closed-loop verification.

[0091] The actual implementation process of step S2 is as follows:

[0092] The UAV visual-language navigation system integrates multimodal navigation input information. The data is fed in parallel into an adaptive hierarchical perception module that integrates VLM-3D fusion for processing. This module associates and infers abstract human intentions, i.e., natural language instructions (such as "fly to the red fire hydrant in the corner of the parking lot"), with complex real-world visual scenes, i.e., real-time image observation sets. It accurately identifies the target red fire hydrant described by the instruction from an image full of distractions (such as red trash cans and traffic cones), and then executes different processes according to the target's state.

[0093] If the target is within the field of view (target lock mode triggered): If the VLM detects a red fire hydrant that matches the navigation task instruction description in the natural language command set in the real-time RGB image observation set, to prevent misidentification of "red traffic cone" as "red fire hydrant," the UAV visual language navigation system performs cross-modal cross-challenge on the image region corresponding to the candidate navigation task target: it reverse-generates an objective image description of the region (e.g., "conical, plastic material"), finds that the objective image description does not semantically match the instruction ("cylindrical, metal"), and determines it as a false detection and removes it. Subsequently, the system determines the number of the remaining true targets:

[0094] If all candidate navigation targets within the current field of view are eliminated, the UAV visual language navigation system determines that the real target is not in the current field of view and automatically switches to semantic search mode.

[0095] If multiple real targets exist within the current field of view, the system determines that the targets are not unique and automatically switches to semantic search mode to find a better observation position.

[0096] If the reverse description matches the command semantics, and only one real target remains within the field of view, after confirming it as a real target (fire hydrant), the UAV visual language navigation system extracts the region from the depth image set. The corresponding depth value is used to reconstruct a local 3D point cloud in real time, combined with camera intrinsic parameters. Then, a density clustering algorithm is used to remove background noise, and the 3D centroid coordinates of the real target are calculated. Projected onto the nearest flyable altitude, the coordinates of the mission endpoint are obtained. Construct an initial set of candidate waypoints with a single element. .

[0097] If the target is not in the field of view (triggers semantic search mode): If the VLM does not detect the target, or if the only real target cannot be identified in the target lock mode, the UAV visual language navigation system calls the VLM to analyze the command using common sense reasoning capabilities, inferring that the target "fire hydrant" is highly likely located in the "sidewalk" or "intersection corner" area. The UAV visual language navigation system, combined with the onboard semantic map, retrieves the positions of the aforementioned potential related landmark areas in three-dimensional space, reconstructs them into local three-dimensional point clouds, calculates the geometric center of each area and projects it to the flight altitude, generating multiple corresponding search guidance points, and filters them based on semantic relevance. Each preferred search guidance point is a single, optimal search point. To avoid the randomness of a single inference, the UAV visual-language navigation system configures and executes separate operations within the semantic constraint area of ​​each preferred search guidance point. Each reasoning process is independent. In each reasoning iteration, VLM provides a reasonably accepted initial 3D waypoint coordinate system. After... After several independent reasoning processes, the UAV visual language navigation system acquires a set of... A set of initial candidate waypoints with different initial waypoints This provides ample options for further in-depth optimization.

[0098] S3: Based on the initial candidate waypoint set and the set self-reflection prompts Obtain the prompt words after injecting candidate waypoints. Then, physical security checks are performed on the candidate waypoints and feedback information is generated. Then, prompt words will be injected after the candidate waypoints. Feedback information With multimodal navigation input information Feed it back into the visual language model The process is repeated to generate corrected candidate waypoints that have undergone serial reflection and optimization. and the revised set of candidate waypoints .

[0099] In this invention, step S3 specifically includes the following steps:

[0100] S31: Unlike traditional single-step decision-making models, this approach targets an initial set of candidate waypoints. Each candidate waypoint Start a The iterative self-correcting process of the wheel;

[0101] S32: In each round of self-correction, the candidate waypoints to be corrected will be... The coordinate values ​​are injected into a preset self-reflective prompt message containing waypoint coordinate correction placeholders. During injection, candidate waypoints are used. Replace the self-reflective prompt with the coordinate values. Correcting placeholders for waypoint coordinates in the middle of the path, and obtaining candidate waypoints for injection. Self-reflection prompts afterward ;

[0102] In this embodiment, self-reflective prompts can be used. The setting is: "Wait, let's reconsider. An initial plan was to move towards [waypoints]. Based on the instruction and current view, let's re-evaluate and provide a better plan.", where [waypoints] is a placeholder for waypoint correction; during injection, candidate waypoints are used. Replace the self-reflective prompt with the coordinate values. The waypoint coordinate correction placeholder [waypoints] in the code;

[0103] A self-reflective prompt is a text instruction template that guides a language model to iteratively self-correct and optimize its output. Unlike traditional one-time questions, self-reflective prompts engage the visual language model... The output generated initially (i.e., a preliminary candidate waypoint) is then used as part of the input again, and the visual language model is instructed using explicit guiding language (such as "Wait, let's reconsider..."). It critically re-evaluates its previous judgments. In this way, the visual language model... They are guided into a cycle of "reflection-optimization" to examine and correct potential flaws in their initial plans.

[0104] S33: For each candidate waypoint The drone visual language navigation system performs physical safety checks, detects the distance between the drone and its surrounding environment, and generates additional natural language feedback based on the check results.

[0105] If the distance between a candidate waypoint and the nearest obstacle is detected to be less than a preset safety threshold, the UAV's visual-language navigation system will convert the specific violation information into a natural language warning and inject it into the base template. For example, if it detects that the distance to an obstacle is too close, it will provide feedback. Set to: "CRITICAL WARNING: The minimum safedistance to the target point has exceeded the safety threshold set at this time. You MUST adjust the path to avoid obstacles."

[0106] If physical detection shows that the safe distance requirement is met, the UAV's visual language navigation system generates positive feedback, guiding the visual language model to further optimize visual alignment details. At this point, the feedback information... Set to: "Thesafety check passed. Please focus on refining the visual alignment with the target description."

[0107] S34: The UAV visual language navigation system selects candidate waypoints The corresponding original multimodal navigation input information Inject candidate waypoints Self-reflection prompts afterward and security feedback information Combined, and then fed back into the visual language model. The process is then performed to generate revised candidate waypoints that have undergone reflection and optimization. ;

[0108] S35: Perform the above method on all candidate waypoints. Through iterative improvements, the UAV visual language navigation system incorporates all rethinking and optimized candidate waypoints. This will lead to a higher quality set of corrected candidate waypoints. .

[0109] The aforementioned iterative correction refers to the process in which, in each subsequent round of self-correction, the visual language model refines the results of the previous round's self-correction. Generated corrected candidate waypoints after serial reflection optimization This serves as the current candidate waypoint for correction, and the process is repeated to perform self-correction.

[0110] In this invention, the above steps enable in-depth reflection and self-correction of each initial candidate waypoint generated through parallel exploration, thereby improving the quality and safety of the solution. That is, for the initial candidate waypoint set... For each initial candidate waypoint, the UAV visual language navigation system initiates an iterative self-correction loop, which executes a total of [number] times. In each iteration, the UAV visual language navigation system performs a "question-reflection" operation: it strings the coordinates of the current waypoint and injects them into a pre-defined template containing guiding language such as "Wait, let's reconsider..." This, along with feedback from physical safety checks, constructs dynamic prompts with clear constraints. It requires no additional training or modification to the VLM; through prompt engineering alone, it can inject candidate waypoints... The template after Together with candidate waypoints The corresponding real-time multi-view images and natural language commands are then fed back into the VLM for processing. Upon receiving the instruction to reflect, the VLM is guided into a deeper reflection loop, enabling it to proactively identify and self-correct any undetected safety or path defects in its initial plan. This process forces the UAV visual-language navigation system to perform more in-depth and robust planning before outputting the final answer, resulting in a more comprehensive, safer, and more reliable corrected candidate waypoint. .go through Through repeated iterations, the UAV visual language navigation system will eventually arrive at a refined set of candidate waypoints after careful consideration. .

[0111] S4: Refine the candidate waypoint set Corrected candidate waypoints Perform multi-dimensional quantitative scoring to obtain corrected candidate waypoints. Multidimensional quantitative scores;

[0112] In this invention, each carefully considered revised candidate waypoint undergoes an objective and quantitative performance evaluation from multiple key dimensions, including safety, target consistency, and forward efficiency. The resulting multi-dimensional quantitative scores provide accurate data support for the final decision.

[0113] In this invention, step S4 specifically includes the following steps:

[0114] S41: Construct a multi-dimensional scoring function;

[0115] For the revised candidate waypoint set Each corrected candidate waypoint A fifth-order polynomial interpolation method constrained by dynamic boundaries is used to construct a path connecting the current position of the UAV. With the target point The smoothed spacetime curve is denoted as the nominal trajectory. The trajectory analysis revealed the drone's trajectory in the future. The full state vector within, including the current position. ,speed angular velocity acceleration and the body attitude derived from the trajectory tangent direction. ;

[0116] Define a function to compute each corrected candidate waypoint Weighted summation formula for fractions :

[0117] ;

[0118] in, , and Based on nominal trajectory The calculated spatiotemporal probability risk score, VLM attention evidence score, and kinetic energy consumption ratio score, , and These are the corresponding weight values, with preset values ​​of 0.5, 0.3, and 0.2, respectively, to prioritize the safety of the drone.

[0119] S42: Constructing a Spatiotemporal Probabilistic Risk Score function;

[0120] Specifically:

[0121] The spatiotemporal probabilistic risk scoring function employs an exponential decay function constructed based on the expected collision risk and a set risk aversion coefficient. The expected collision risk is constructed based on the degree of overlap between the UAV's probabilistic existence pipeline and the obstacle existence probability density field in the spatiotemporal domain. The probabilistic existence pipeline is constructed based on the UAV's three-dimensional position coordinates at each moment, the expected position vector of the nominal trajectory at the corresponding moment, and the covariance matrix of the position estimation error at that moment; the covariance matrix includes process noise accumulated over time. The obstacle existence probability density field is constructed by superimposing Gaussian kernel functions corresponding to each obstacle point in the local spatial point cloud. The nominal trajectory is a smooth spatiotemporal curve connecting the corrected candidate waypoints, the UAV's current position, and the target point.

[0122] First, the multi-view depth sensing equipment onboard the drone is used to acquire the three-dimensional geometric information of the environment and generate a local spatial point cloud. Then, kernel density estimation (KDE) is used to transform the point cloud into a continuous, differentiable probability density function. For any point in space... The probability density of the existence of obstacles Defined as a local spatial point cloud All the clouds The formula for the superposition of Gaussian kernel functions is as follows:

[0123]

[0124] in, It is a 3×3 measurement noise covariance matrix determined by the characteristics of the depth sensor, representing the variance and correlation of the three-dimensional spatial coordinate measurement error. This formula maps the discrete physical environment into a continuous risk potential field.

[0125] For each candidate waypoint to be evaluated Using the route planned in step S41 from the current position Arrive at the candidate waypoint to be evaluated nominal trajectory ,in Given that the state estimation (SLAM / VIO) will drift over time, the drone at future moments... The true location cannot be represented by a single coordinate point, but is modeled as a Gaussian distribution that dilates over time. .in, For nominal trajectory The desired position vector at time t, i.e. the three-dimensional reference coordinates given by the path planning module; The covariance matrix is ​​the position estimation error at that moment, used to represent the three-dimensional positioning uncertainty caused by factors such as drift and IMU noise. Its size and shape reflect the range of possible distribution of the UAV's true position.

[0126] To accurately calculate the covariance matrix The UAV visual language navigation system combines current positioning uncertainty And generated by IMU integration and along with the state transition matrix Propagation process noise Its accumulation process over time can be described by the following integral formula:

[0127]

[0128] in, The integral time variable represents the time from the initial time 0 to the predicted time. Any instant between; Indicates at time The state transition matrix, which is the linearized Jacobian matrix of the UAV's nonlinear kinematic equations at the nominal trajectory point at that moment, describes how process noise evolves with the system dynamics and accumulates on the time axis; The three-dimensional covariance of the position estimate at the initial time; This is the 3×3 covariance matrix of the IMU process noise.

[0129] Based on the above calculations, the drone's motion state over a future period is no longer considered a single deterministic trajectory, but rather modeled as a probabilistic conduit in four-dimensional spacetime. :

[0130]

[0131] in, This represents the three-dimensional position coordinates of the drone. This spatiotemporal structure combines the three-dimensional spatial position with a one-dimensional time dimension, describing the evolution of the drone's potential position as a "probability cloud" surrounding the nominal trajectory, whose coverage area (i.e., the thickness of the pipe) gradually expands over time.

[0132] At this point, collision risk is no longer defined as a single moment of distance contact, but rather as the degree of mutual penetration between the probability density fields of the UAV's probabilistic presence in the conduit and the obstacle's presence throughout the entire spatiotemporal domain. The UAV visual-language navigation system calculates these two probability distributions over the entire time period. and full three-dimensional space The overlap integral is used to quantify the expected collision risk under conditions that consider all perception and control uncertainties. :

[0133]

[0134] The higher the integral value, the greater the probability density that the location of the drone may coincide with the location of the obstacle in space and time, and the higher the potential danger.

[0135] Finally, to transform the unbounded risk integral into a standardized scoring metric, this invention employs an exponential decay function to nonlinearly map the risk value. A risk aversion coefficient is also introduced. This is used to adjust the sensitivity of the UAV's visual language navigation system to risk, ultimately obtaining the spatiotemporal probability risk score for the candidate waypoint. :

[0136]

[0137] The output range of the scoring function is (0, 1], where the score of high-risk paths decreases exponentially, thus forcing UAV visual language navigation systems to prioritize safer paths that are more robust in a probabilistic sense, rather than paths that appear safe only in terms of geometric distance.

[0138] The aforementioned spatiotemporal probability risk scoring design significantly enhances the safety of drones in complex environments. It empowers the drone's visual-language navigation system with the ability to "predict uncertainty," moving beyond mere momentary distance judgments to fully consider potential sensor errors and positional drift during flight. By viewing the drone as a "probability range" moving through time and space rather than a rigid point, this design allows the drone's visual-language navigation system to keenly detect potential dangers even when positioning signals are inaccurate or environmental information is slightly ambiguous. It then plans a robust path capable of handling various interferences, effectively preventing accidental collisions caused by perception biases.

[0139] S43: Constructing VLM Attention Evidence Scoring function;

[0140] Specifically:

[0141] The VLM attention evidence scoring function employs a hyperbolic tangent function constructed based on the total information gain of the key evidence set at the waypoint and a set gain sensitivity coefficient. The key evidence set is constructed based on the image block with the highest weight in the cross-modal attention graph generated by the visual language model and its corresponding 3D surface normal vector. The total information gain is constructed based on the combined effect of the geometric perspective gain of each evidence block in the key evidence set. The geometric perspective gain is constructed based on the relationship between the alignment of the observation direction at the future time and the observation direction at the current time with respect to the target surface normal.

[0142] The UAV visual language navigation system delves into the Transformer attention layer of the visual language model to obtain the currently input natural language commands. With real-time image observation A cross-modal attention map is generated between the images. This map represents the importance of each patch in the image for the visual language model to understand the instructions. The UAV visual language navigation system ranks the attention weights and selects the image patches with the highest weights to construct a key evidence set. This is equivalent to locking onto those pixel areas that the VLM is "looking at" at the current moment in order to confirm "what this is" (such as the red surface of a fire hydrant or the unique nozzle structure).

[0143] against Each key piece of evidence The UAV visual language navigation system combines the corresponding depth image information to back-project the pixel coordinates of the block into three-dimensional space to obtain the surface normal vector of the block. Subsequently, a composite quality evaluation function is constructed. The composite quality evaluation function is determined by both texture sharpness and geometric observability. First, the key evidence blocks are calculated using the Laplacian operator. The image grayscale gradient variance within the range is used to obtain texture sharpness. First, to penalize ambiguous areas; second, to calculate the observation vector pointing from the current UAV position to the three-dimensional coordinates of the evidence block. With surface normal vector The degree of alignment is used to obtain the geometric observability at the current moment. The formula is defined as follows:

[0144]

[0145] The overall quality function is:

[0146]

[0147] in, The normalization coefficient is... This represents the original attention weight for this block. This formula quantifies the effectiveness of the evidence information from the current perspective.

[0148] The UAV visual language navigation system simulates the UAV following candidate waypoints Virtual pose (position) after movement ,attitude (and recalculate all key evidence blocks in that virtual pose), The expected observation quality. This process involves two parts of gain calculation:

[0149] Geometric viewpoint gain: Calculates the change in the angle between the new observation vector and the surface normal vector to obtain the geometric observability at future time points. .

[0150] Taking into account the above factors, the total information gain of candidate waypoints to the key evidence set is calculated. :

[0151]

[0152] in, and , respectively, represent the Euclidean distances between the drone and evidence point b at the current and future times. , For weight parameters, To prevent tiny positive numbers with a denominator of zero.

[0153] Finally, to map the unbounded gain value to a standardized scoring range and to simulate the saturation effect of perception (i.e., the score increase should slow down after sufficient visibility), a hyperbolic tangent function is used to nonlinearly activate the total gain. This ultimately yields the VLM attention evidence score for the candidate waypoint. :

[0154]

[0155] in, This is the gain sensitivity coefficient. This scoring mechanism makes the drone tend to choose paths that it can "see more clearly, more accurately, and closer," thereby actively optimizing the signal-to-noise ratio of the VLM input and improving the confidence of subsequent decisions.

[0156] The aforementioned VLM attention evidence scoring design endows the drone with a proactive verification capability, enabling it to fly not only to reach the destination but also to clearly see the target. By guiding the drone to prioritize routes that make key objects in the command appear larger, clearer, and at a more accurate angle in the field of vision during path planning, this design ensures that the drone's visual language navigation system continuously collects high-quality visual evidence during movement. This allows the drone to eliminate visual ambiguity and uncertainty by actively adjusting its attitude, much like an experienced observer, ensuring that every decision is based on what is seen, greatly enhancing the accuracy and confidence of navigation mission execution.

[0157] S44: Constructing a dynamic energy consumption ratio score function;

[0158] Kinetic energy consumption ratio score The function is based on the ratio of the effective progress of the UAV's position to the total energy cost functional of the UAV's flight along the trajectory. The effective progress is constructed based on the effective propulsion component of the displacement direction of the candidate trajectory in the global target direction. The total energy cost functional is constructed based on the cumulative instantaneous total power of the UAV over the entire time period. The instantaneous total power is constructed based on the induced power required to maintain hovering and lift, the parasitic power to overcome air resistance, and the control power generated by attitude adjustment. The induced power, parasitic power, and control power are determined by the rotor total thrust vector, the instantaneous velocity, and the instantaneous angular velocity vector of the UAV, respectively, calculated based on the inverse dynamics model.

[0159] Specifically:

[0160] First, the acceleration vector obtained by resolving the nominal trajectory generated in step S31 is used. Combined with the physical mass of the drone With the gravitational acceleration vector The total thrust vector that a UAV rotor must generate is calculated by inversely solving the Newton-Euler equations. This thrust not only needs to counteract the gravitational component but also needs to provide the inertial force required for maneuvering, as expressed by the formula:

[0161]

[0162] This process transforms the geometric trajectory into the demand for physical control forces, providing a mechanical basis for subsequent power estimation.

[0163] Based on the characteristics of brushless DC motors and rotor aerodynamics theory, an instantaneous total power model is constructed. The model comprehensively considers three main power consumption components: the induced power required to maintain hover and lift (proportional to the 3 / 2 power of thrust), the parasitic power to overcome air resistance (proportional to the cube of velocity), and the control power generated by attitude adjustment (proportional to the square of angular velocity). The UAV visual-language navigation system provides full-time-cycle... Integrating the instantaneous power over time yields the total energy cost functional for flight along the trajectory. :

[0164]

[0165] in, , , These are dynamic coefficients related to air density, blade geometry, and motor efficiency, respectively. Indicates the instantaneous speed of the drone. This represents the instantaneous angular velocity vector of the drone's body; this integral formula precisely quantifies the energy cost required for the drone to perform this action.

[0166] To eliminate invalid wandering or lateral drift, a "valid progress" relative to the final goal must be defined. The UAV visual-language navigation system acquires corrected candidate waypoints. Build from the current location of the drone Point to the global final target point specified in the natural language instructions for the navigation task. global target vector Then, the displacement vector of the candidate trajectory is calculated. The orthogonal projection component along the global objective vector direction is defined as the effective progress. :

[0167]

[0168] in, To prevent the use of tiny positive numbers with a denominator of zero, this calculation uses a vector dot product operation to geometrically filter out invalid components perpendicular to the target direction, retaining only the displacements that substantially contribute to task completion.

[0169] Finally, to find the optimal balance between energy consumption and mission progress, an evaluation system for effective progress per joule was constructed. The final kinetic-energy ratio score was obtained by dividing the effective progress by the regularized energy cost. :

[0170]

[0171] in, To prevent denominator singularity, a regularization constant is used. The scoring function is a highly nonlinear filter that automatically penalizes schemes that require drastic maneuvers (leading to a surge in power spikes) or circuitous paths (resulting in low effective progress), guiding the UAV to choose a streamlined trajectory that is both hydrodynamically smooth and efficiently direct to the target.

[0172] The aforementioned dynamic energy efficiency ratio design incorporates a physical performance evaluation dimension, enabling the UAV's visual-language navigation system to intelligently plan flight from an energy utilization perspective. By comprehensively calculating the relationship between motor thrust, air resistance, and mission progress, this design can automatically select high-quality paths that efficiently approach the target while maximizing energy savings. This not only avoids energy waste and flight instability caused by violent maneuvers but also ensures that the UAV performs its mission in the most economical and smoothest manner, thereby significantly extending operational time and improving overall flight smoothness within limited battery life.

[0173] S45: Using the multi-dimensional scoring function constructed in step S41, calculate the corrected candidate waypoint for each waypoint. Multidimensional quantitative scores;

[0174] That is, to modify the set of candidate waypoints Each corrected candidate waypoint The obtained corrected candidate waypoint Spatiotemporal probability risk score VLM Attention Evidence Score and kinetic energy consumption ratio score Substituting these values ​​into the multi-dimensional scoring function, the multi-dimensional quantitative score of the corrected candidate waypoint is calculated. ;

[0175] In this invention, before selecting the optimal waypoint, the UAV visual-language navigation system first performs a deep spatiotemporal probabilistic risk assessment on each corrected waypoint. The system utilizes a depth camera to construct an environmental obstacle field and combines this with sensor errors to predict the potential positional drift range of the UAV during flight. By calculating the spatiotemporal overlap between this drift range and obstacles, the system derives a safety score that encompasses dynamic uncertainties. This design enables the UAV to anticipate dangers, allowing it to move beyond relying solely on current static distances and proactively avoid potential collisions caused by inaccurate positioning or environmental interference, prioritizing safe paths that are robust enough to withstand various unforeseen circumstances.

[0176] In this invention, after completing the safety assessment, the UAV visual language navigation system evaluates the observation quality of each corrected waypoint for key visual evidence from an active perception perspective. Specifically, the UAV visual language navigation system delves into the internal structure of the visual language model, extracting the core image regions (such as the color or shape of a fire hydrant) that the model focuses on for target identification, and combines this with three-dimensional geometric information to determine whether candidate waypoints allow the UAV to see these regions more clearly at closer distances and from a more accurate angle. This scoring rewards waypoints that significantly improve visual clarity and information acquisition, ensuring that every step the UAV takes is for collecting more conclusive visual evidence, thereby avoiding disorientation or misjudgment due to blurred perspective.

[0177] In this invention, to ensure that the UAV balances endurance and efficiency during mission execution, the UAV visual-language navigation system also performs a dynamic energy efficiency assessment at each corrected waypoint. Based on first principles of physics, it simulates and calculates the actual power consumption required for the UAV to overcome gravity and air resistance and perform specific flight maneuvers, comparing this energy cost with the effective displacement made by the UAV towards the target. The resulting energy consumption ratio score can automatically filter out paths that, although geometrically short, require intense maneuvers that would lead to a surge in power consumption, guiding the UAV to choose a streamlined trajectory that is smooth, saves power to the maximum extent, and efficiently approaches the target.

[0178] S5: Multi-dimensional quantitative score based on each acquired corrected candidate waypoint For all candidate waypoints to be corrected The quantitative scores are summarized and settled. The corrected candidate waypoint with the highest multi-dimensional quantitative score is selected as the optimal waypoint. A fine-grained path decoder of multilayer perceptron (MLP) is used to generate a fine trajectory containing a continuous pose sequence. The UAV is controlled to reach the target and complete the mission based on the fine trajectory.

[0179] In this invention, step S5 includes the following steps:

[0180] S51: Select the corrected candidate waypoint with the highest multi-dimensional quantization score as the optimal waypoint;

[0181] Compare the multi-dimensional quantization scores of all revised candidate waypoints, and select the maximum score to determine the corresponding optimal waypoint. :

[0182] ;

[0183] S52: Employing a hierarchical trajectory generation architecture, the optimal waypoints determined in step S51 are... Inputting a fine-grained path decoder based on a multilayer perceptron (MLP), the UAV visual-language navigation system, under the premise of satisfying physical obstacle avoidance constraints, fuses real-time acquired forward-view image features with optimal waypoint coordinate features (in a visible target scene, this point coincides with the mission endpoint) to regress and generate a fine trajectory containing a continuous pose sequence. .

[0184] S53: Develop the fine trajectory sequence generated in step S52. As the final result of this decision-making cycle, navigation commands are output to control the UAV to proceed to the next flight step according to the optimal waypoint.

[0185] In this embodiment, when the fine trajectory sequence Once the final navigation command is determined, it is sent to the UAV's Flight Control System (FCS). The FCS is the core of the UAV hardware, responsible for translating high-level navigation objectives into low-level, executable motor control signals. The FCS can calculate in real time the roll, pitch, yaw, and throttle commands required to reach the target waypoint, and precisely control the rotational speed of each rotor, thereby driving the UAV to perform the corresponding maneuvers in the physical world. This step completes a closed loop from perception and decision-making to physical execution, enabling the entire navigation mission to proceed in a simulated environment, while simultaneously achieving the execution of navigation commands and closed-loop control.

[0186] Example:

[0187] This embodiment simulates the complete process of a drone generating the next navigation path based on the current view image and target commands through a visual language model.

[0188] S1: After receiving the navigation task trigger command, the UAV acquires image and depth data, and combines this with the natural language commands given by the user regarding the navigation task objective to obtain the UAV's multimodal navigation input information. ;

[0189] in, These represent the real-time image observation sets collected by five cameras mounted on the drone from five different perspectives (front, rear, left, right, and downward). Indicates and A real-time depth image set with one-to-one viewpoint correspondence. Represents a natural language command issued by the user describing the target of the navigation task: "There is a target in the right front of uav. Using your front as the x-axis and your right as the y-axis, The target is at a yaw angle of 38.0 degrees from you. The red fire hydrant is located on a paved sidewalk in an urban environment with a mix of tall buildings, some with notable architectural details, andwide streets. Various street objects such as traffic cones, trash bins, and poles are present nearby, along with scattered vegetation and parkedvehicles, providing clear visual landmarks to help identify the location. Please control the drone and find the target."

[0190] S21: Input multimodal navigation information The image is fed into the VLM-3D fusion adaptive hierarchical perception module; the UAV visual language navigation system uses open vocabulary target detection technology to associate and reason with the natural language command and the real-time RGB image observation set to determine whether there is a candidate navigation task target in the current field of view that matches the description of the navigation task command; after the detection and image viewing, although there are objects such as red traffic cones in the current field of view, after cross-modal challenge, it is determined that there is no sure target in the current field of view that completely matches the semantics of the command "red fire hydrant". Therefore, the UAV visual language navigation system automatically enters the semantic search mode and proceeds to step S23.

[0191] S23: Visual language models analyze instructions using common-sense reasoning abilities. The system infers that the target is highly likely located in an associated landmark area such as "paved sidewalk" or "street corner". The UAV visual-language navigation system retrieves the semantic tags corresponding to these areas from the semantic map's tag library, finding the semantic tag Class ID=15 for "paved sidewalk" and Class ID=21 for "street corner". Within the sector forward of the UAV's current position (yaw angle ±45°), voxels labeled with the above IDs are retrieved and clustered. The geometric center of each associated area in 3D space is calculated and projected onto the flight altitude, thereby generating multiple potential search guidance points. This embodiment demonstrates the optimal number of guidance points. In the case of =1: The search guidance points are sorted according to semantic relevance, and the center of the “paved sidewalk” region is finally selected as the preferred search guidance point, with coordinates [0.75, 0.03, -4.80], which serves as the general direction for guiding the search.

[0192] S24: Based on the real-time image observation set and natural language instructions for the navigation task objective, several candidate waypoints are generated in parallel within the semantically constrained region, and an initial candidate waypoint set is constructed. This embodiment demonstrates... In the case of =3, the UAV visual-language navigation system generates three initial candidate waypoints in parallel based on this multimodal input: = [0.72, 0.03, -4.82]; = [0.85, 0.02, -4.75]; = [0.69, 0.04, -4.90]. These three waypoints together constitute the initial candidate waypoint set. .

[0193] S31: For the initial set of candidate waypoints For each candidate waypoint in the list, initiate a The iterative self-correcting process of the wheel is demonstrated in this embodiment. When the value is 1, a reflexive correction is performed.

[0194] S32: In each round of self-correction, the candidate waypoints to be corrected will be... The coordinate values ​​are injected into a preset self-reflective prompt message that includes waypoint coordinate correction placeholders. In: "Wait, let's reconsider. An initial plan was to move towards [waypoints]. Based on the instruction and current view, let's re-evaluate and provide a better plan."

[0195] Post-injection prompt As shown below:

[0196] : "Wait, let's reconsider. An initial plan was to move towards [0.72, 0.03, -4.82]. Based on the instruction and current view, let's re-evaluate and provide a better plan."

[0197] : "Wait, let's reconsider. An initial plan was to move towards [0.85, 0.02, -4.75]. Based on the instruction and current view, let's re-evaluate and provide a better plan."

[0198] : "Wait, let's reconsider. An initial plan was to move towards [0.69, 0.04, -4.90]. Based on the instruction and current view, let's re-evaluate and provide a better plan."

[0199] S33: The UAV visual language navigation system utilizes real-time depth maps Calculate the minimum obstacle distance around each candidate point. Set a safety threshold. = 0.5m. Feedback information Set to:

[0200] : =1.2 m > , feedback information: "The safety check passed. Please focus on refining the visual alignment with the target description."

[0201] : =0.6 m > , feedback information: "The safety check passed. Please focus on refining the visual alignment with the target description."

[0202] : =0.4 m < , feedback information: "CRITICAL WARNING: The minimum safedistance to the target point has fallen below the safety threshold set at this time. You MUST adjust the path to avoid obstacles."

[0203] S34: The UAV visual language navigation system selects candidate waypoints The corresponding original multimodal navigation input information, and the self-reflective prompts after injecting candidate waypoints. Combine the text. The combined text input is shown below:

[0204] :“A chat between a curious user and an artificial intelligenceassistant. The assistant is a navigation model that output the uav waypointsaccording to the user's instructions. USER: Stage:take off\n\nPreviousdisplacement:0.0,0.0,-1.0\n\nCurrent position:0.0,0.0,0.0\n\nCurrent image: \n\nInstruction: There is a target in the right front of uav.Using yourfront as the x-axis and your right as the y-axis, The target is at a yawangle of 38.0 degrees from you. The red fire hydrant… find the target.\nWait, let's reconsider. An initial plan was to move towards [0.72, 0.03,-4.82]. Based on the instruction and current view, let's re-evaluate andprovide a better plan. The safety check passed. Please focus on refining thevisual alignment with the target description.”

[0205] : "A chat between ... find the target.\nWait, let's reconsider. Aninitial plan was to move towards [0.85, 0.02, -4.75]. Based on the instruction and current view, let's re-evaluate and provide a better plan. The safety check passed. Please focus on refining the visual alignment with the target description."

[0206] : "A chat between ... find the target.\nWait, let's reconsider. Aninitial plan was to move towards [0.69, 0.04, -4.90]. Based on the instruction and current view, let's re-evaluate and provide a better plan. CRITICAL WARNING: The minimum safe distance to the target point has fallenbelow the safety threshold set at this time. You MUST adjust the path to avoid obstacles."

[0207] The combined multimodal navigation inputs are fed into the visual language model. The process is then performed to generate revised candidate waypoints that have undergone reflection and optimization. = [ 0.66, 0.08, -4.86 ]; = [ 0.83, 0.02, -4.78 ]; = [0.69, 0.05, -4.91].

[0208] S35: Perform the above method on all candidate waypoints. =1 round of iterative correction; the UAV visual language navigation system will reflect on and optimize all candidate waypoints for correction. This will lead to a higher quality set of corrected candidate waypoints. .

[0209] S41: Construct a multi-dimensional scoring function; define a function to calculate each corrected candidate waypoint. Weighted summation formula for fractions :

[0210] ;

[0211] in, , and Based on nominal trajectory The calculated spatiotemporal probability risk score, VLM attention evidence score, and kinetic energy consumption ratio score, , and These are the corresponding weight values, with preset values ​​of 0.5, 0.3, and 0.2, respectively, to prioritize the safety of the drone.

[0212] S42: Calculate the spatiotemporal probability risk score ;

[0213] The UAV visual language navigation system provides parallel services for Spatiotemporal probabilistic risk scores are calculated for the three corrected candidate waypoints. An obstacle probability density field for the current environment is constructed using kernel density estimation, and the collision probability integral is obtained. and its corresponding The scores are as follows:

[0214] for The corrected waypoints avoided major obstacle zones during descent, and the contraction of the x-axis increased the safety margin. The collision probability integral was calculated. ≈ 0.05, score after exponential mapping ≈0.95;

[0215] for Despite braking corrections, x=0.83 still indicates the foremost position, closest to the obstacle, and thus carries a relatively high risk. The collision probability integral is calculated. ≈ 0.20, score after exponential mapping ≈0.82;

[0216] for The most conservative location carries the lowest risk. The collision probability integral is calculated. ≈ 0.04, score after exponential mapping ≈0.96.

[0217] S43: Calculate the VLM attention evidence score ;

[0218] Extract the attention map of the VLM for the instruction keyword "red fire hydrant" and calculate the geometric observability gain by combining it with coordinates. The UAV visual-language navigation system provides parallel services for... The VLM attention evidence score is calculated from the three corrected candidate waypoints, and the corresponding... The scores are as follows:

[0219] for The rightward shift correction of y=0.08 better aligned the viewpoint with the target to the right front, and the depth z=-4.86 ensured a suitable distance and the clearest texture. (Score calculation) =0.88;

[0220] for The y=0.02 parameter indicates almost no lateral movement, resulting in a skewed observation angle of the target to the right front, leading to incomplete information acquisition. (Score calculation follows.) =0.70;

[0221] for The observation angle is acceptable, but the location is somewhat remote. (Score calculated) =0.85.

[0222] S44: Calculate the dynamic energy consumption ratio score ;

[0223] Based on the inverse solution of the Newton-Euler equations, the total rotor thrust and air resistance power consumption required to maintain the nominal trajectory are calculated. The total energy consumption integral over the entire flight cycle is compared with the effective propulsion component of the waypoint in the target direction, thereby quantitatively evaluating the energy utilization efficiency of that waypoint. The UAV visual-language navigation system provides parallel support for... The dynamic energy ratio scores of the three corrected candidate waypoints are calculated, and the corresponding scores are... The scores are as follows:

[0224] for Although it involves a vertical displacement of 4.8m, requiring significant power for ascent and descent control, this waypoint effectively advanced the mission, and the trajectory was smooth without sharp turns. (Score calculated) =0.90;

[0225] for This waypoint has the shortest Euclidean distance and almost no lateral displacement (y=0.02), greatly reducing the rotational power consumption for attitude adjustment. The overall travel distance is short and the efficiency is high. (Score calculated) =0.84;

[0226] for The vertical displacement is the largest (-4.91), and the total power consumption is slightly higher. Calculation score. =0.88.

[0227] S45: Using the multi-dimensional scoring function constructed in step S31, calculate the corrected candidate waypoint for each waypoint. Multidimensional quantitative scores;

[0228] That is, to modify the set of candidate waypoints Each corrected candidate waypoint The obtained spatiotemporal probability risk score VLM Attention Evidence Score and kinetic energy consumption ratio score Substituting these values ​​into the multi-dimensional scoring function, the multi-dimensional quantitative score of the corrected candidate waypoint is calculated:

[0229] =0.5×0.95+0.3×0.88+0.2×0.90=0.919;

[0230] =0.5×0.82+0.3×0.70+0.2×0.84=0.788;

[0231] =0.5×0.96+0.3×0.85+0.2×0.88=0.911.

[0232] S51: Select the corrected candidate waypoint with the highest multi-dimensional quantization score as the optimal waypoint;

[0233] Compare the multi-dimensional quantization scores of all corrected candidate waypoints and select the maximum value. To determine the corresponding optimal waypoint = [0.66, 0.08, -4.86];

[0234] S52: Unmanned Aerial Vehicle Visual Language Navigation System will The input path decoder fuses features from the foreground image and regresses to generate a fine trajectory containing a continuous pose sequence. .

[0235] S53: Fine trajectory As the final result of this decision-making cycle, navigation commands are output to control the UAV to proceed to the next flight step according to the optimal waypoint.

[0236] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the following claims.

Claims

1. A test-time extended UAV navigation method based on semantic and physical perception, characterized in that, Includes the following steps: S1: The UAV acquires image and depth data and combines it with the navigation mission objectives in the received navigation mission instructions to construct the UAV's multimodal navigation input information; S2: Based on the multimodal navigation input information, determine whether there exists a unique candidate navigation task target within the current field of view that matches the navigation task instruction description: If so, determine the three-dimensional position of the unique candidate navigation mission target and construct an initial set of candidate waypoints containing only a single, certain navigation mission target; If not, identify multiple potentially related landmark areas in the current field of view that are relevant to the navigation task objective, and generate several initial candidate waypoints in parallel based on the center of each potential related landmark area, and construct an initial candidate waypoint set containing multiple initial candidate waypoints; S3: Based on the initial set of candidate waypoints and the set self-reflective prompts, obtain the prompts after injecting them into the candidate waypoints; then perform physical security checks on the candidate waypoints to generate feedback information; The prompts, feedback information, and multimodal navigation input information injected after the candidate waypoints are then fed back into the visual language model to generate corrected candidate waypoints and a set of corrected candidate waypoints after serial reflection and optimization. S4: Perform multi-dimensional quantitative scoring on the corrected candidate waypoints in the corrected candidate waypoint set; S5: Select the corrected candidate waypoint with the highest multi-dimensional quantization score as the optimal waypoint, and use the optimal waypoint to generate a trajectory control UAV containing a continuous pose sequence.

2. The method according to claim 1, characterized in that, Step S2 specifically includes the following steps: S21: Using open vocabulary target detection technology, the natural language instructions and real-time RGB image observation set are correlated and inferred to determine whether there are candidate navigation task targets in the current field of view that match the description of the navigation task instructions. If one or more candidate navigation task targets are detected in the current field of view, then enter the target locking mode and execute step S22; if no candidate navigation task targets are detected in the current field of view, then enter the semantic search mode and execute step S23. S22: Perform semantic matching judgment on each candidate navigation task target; if the semantics do not match, the candidate navigation task target is determined to be a false detection and is removed; if the semantics match, the current candidate navigation task target is retained as the real target. Once all candidate navigation task targets within the field of view have been matched, determine whether only one true target remains. If there is only one real target, determine the three-dimensional position of the real target and then proceed to step S24; if there is no real target or there are multiple real targets, proceed to step S23. S23: Using semantic association reasoning, identify multiple potential associated landmark regions that are relevant to the navigation task target from the current real-time RGB image observation set, and calculate the center of each associated landmark region to generate multiple corresponding search guidance points; then select several preferred search guidance points based on semantic relevance. S24: If in target locking mode, the mission endpoint coordinates are directly encapsulated into an initial candidate waypoint set containing only a single element; If in semantic search mode, then by combining the real-time image observation set and the navigation task objective, and using a visual language model, N initial candidate waypoints are generated in parallel for each of the K preferred search guide points within the semantic constraint area of ​​the K preferred search guide points, thus constructing an initial candidate waypoint set.

3. The method according to claim 1, characterized in that, Step S3 specifically includes the following steps: S31: For each candidate waypoint in the initial candidate waypoint set, perform an M-round iterative self-correction process; S32: In each round of self-correction, the coordinate values ​​of the candidate waypoints are used to replace the waypoint coordinate correction placeholders in the self-reflective prompts, resulting in the self-reflective prompts after injecting the candidate waypoints; S33: For each candidate waypoint, detect the distance between the drone and the surrounding environment, and generate additional natural language feedback based on the detection results; S34: Combine the original multimodal navigation input information corresponding to the candidate waypoint, the self-reflective prompts and safety feedback information after injecting the candidate waypoint, and send them back to the visual language model for processing to finally generate the corrected candidate waypoints after reflection and optimization. S35: Following the above method, all candidate waypoints are iteratively corrected for M rounds, and all corrected candidate waypoints that have been reflected and optimized are gathered to form a corrected candidate waypoint set.

4. The method according to claim 1, characterized in that, Step S4 specifically includes the following steps: S41: Construct a multi-dimensional scoring function formed by weighted summation of spatiotemporal probabilistic risk, VLM attention evidence, and dynamic energy consumption ratio; S42: Construct a spatiotemporal probabilistic risk scoring function; S43: Construct the VLM attention evidence scoring function; S44: Construct a dynamic energy consumption ratio scoring function; S45: Calculate the multi-dimensional quantitative score for each corrected candidate waypoint using a multi-dimensional scoring function.

5. The method according to claim 1, characterized in that, In step S22: cross-modal challenge is performed on the image region, and an image description of the region is generated in reverse and semantically compared with the navigation task instruction; if the generated image description matches the semantics of the navigation task instruction, the candidate navigation task target is determined to be consistent with the navigation task instruction description and confirmed as the real target. If the reverse-generated image description does not match the semantics of the navigation task instruction, then the candidate navigation task target is determined to be inconsistent with the navigation task instruction description, and the candidate navigation task target is a false detection.

6. The method according to claim 1, characterized in that, In step S22: After confirming the unique real target, the depth data corresponding to the real target area in the real-time depth image set is extracted, and the real target is reconstructed into local three-dimensional point cloud data in real time by combining the camera intrinsic parameters. Subsequently, a density clustering algorithm is used to calculate the three-dimensional centroid coordinates of the real target. The three-dimensional centroid coordinates are then projected onto the nearest flyable altitude and recorded as the mission endpoint coordinates as a unique element for temporary storage.

7. The method according to claim 1, characterized in that, In step S23: First, a visual language model is used to infer potential associated landmark areas that are relevant to the target; then, combined with an airborne semantic map, the positions of potential associated landmark areas in the current field of view in three-dimensional space are retrieved, and local three-dimensional point clouds are constructed respectively; then, a density clustering algorithm is used to calculate the three-dimensional centroid coordinates of the local three-dimensional point cloud; then, these three-dimensional centroid coordinates are projected to the nearest flightable altitude to generate multiple corresponding search guidance points; finally, K preferred search guidance points are selected based on semantic relevance.

8. The method according to claim 4, characterized in that: The spatiotemporal probabilistic risk scoring function adopts an exponential decay function constructed based on the expected collision risk and a set risk aversion coefficient. The expected collision risk is constructed based on the degree of overlap risk between the UAV's probabilistic existence pipeline and the probability density field of obstacle existence in the spatiotemporal domain. The probabilistic existence pipeline is constructed based on the UAV's three-dimensional position coordinates at each time, the expected position vector of the nominal trajectory at the corresponding time, and the covariance matrix of the position estimation error at that time. The covariance matrix contains process noise accumulated over time. The probability density field of obstacle existence is constructed by superimposing the Gaussian kernel functions corresponding to each obstacle point in the local spatial point cloud. The nominal trajectory is a smooth spatiotemporal curve connecting the corrected candidate waypoint, the UAV's current position, and the target point.

9. The method according to claim 4, characterized in that: The VLM attention evidence scoring function employs a hyperbolic tangent function constructed based on the total information gain of the key evidence set at the waypoint and a set gain sensitivity coefficient. The key evidence set is constructed based on the image block with the highest weight in the cross-modal attention graph generated by the visual language model and its corresponding 3D surface normal vector. The total information gain is constructed based on the combined effect of the geometric perspective gain of each evidence block in the key evidence set. The geometric perspective gain is constructed based on the relationship between the alignment of the observation direction at the future time and the observation direction at the current time with respect to the target surface normal.

10. The method according to claim 4, characterized in that: The dynamic energy consumption ratio scoring function is the ratio of the effective progress based on the UAV's position to the total energy consumption cost functional of the UAV flying along the trajectory. The effective progress is constructed based on the effective propulsion component of the displacement direction of the candidate trajectory in the global target direction. The total energy consumption cost functional is constructed based on the cumulative instantaneous total power of the UAV over the entire time period. The instantaneous total power is constructed based on the induced power required to maintain hovering and lift, the parasitic power to overcome air resistance, and the control power generated by attitude adjustment. The induced power, parasitic power, and control power are determined by the rotor total thrust vector, the instantaneous velocity, and the instantaneous angular velocity vector of the UAV, respectively, calculated based on the inverse dynamics model.

Images