A method and system for semantic navigation of unmanned aerial vehicles based on large models and reinforcement learning

Abstract

The application discloses a kind of unmanned aerial vehicle semantic navigation method and system based on large model and reinforcement learning, it is related to the technical field of artificial intelligence, for the problems that existing technology cannot effectively execute fuzzy natural language instruction and large model is difficult to directly carry out bottom layer real-time control, the application adopts layered decoupling architecture: first, natural language instruction and semantic map are input into first multi-modal large model to extract or infer semantic target;Second, high-level motion direction is output by second multi-modal large model in combination with multi-direction image;Then, in local feasible region, the integrated weight is calculated by fusing historical trajectory, large model direction, safety distance and semantic distance, and the local navigation target point is selected;Finally, the target point and environmental depth features are input into SAC reinforcement learning model, and continuous control instruction is output to drive unmanned aerial vehicle.The application effectively bridges the gap between high-level semantic reasoning and bottom layer real-time control, and realizes safe and efficient natural language driven navigation in complex environment.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and more specifically, to a semantic navigation method and system for unmanned aerial vehicles (UAVs) based on large models and reinforcement learning. Background Technology

[0002] Unmanned aerial vehicles (UAVs) have broad application prospects in fields such as search and rescue, industrial inspection, and logistics delivery. Traditional UAV navigation methods mostly rely on preset waypoints or map-based geometric planning. Their core limitation is that they cannot understand vague and abstract human natural language commands (such as "find me some plastic pipes"), making it difficult to perform flexible search tasks in unknown environments.

[0003] In recent years, researchers have attempted to introduce deep reinforcement learning (DRL) or large language models (LLM) into the field of UAV navigation. However, pure DRL methods are inefficient in handling high-level semantic understanding and multi-step planning tasks in sparse reward environments, and they are difficult to directly process natural language input. On the other hand, if large models are used directly for the low-level control of UAVs, although they can understand language commands, they face prominent problems such as high computational overhead, high inference latency, and difficulty in generating continuous low-level motion control signals.

[0004] Therefore, how to effectively combine high-level semantic understanding with low-level continuous motion control, overcome the contradiction between large model computation latency and the real-time control requirements of UAVs, and enable UAVs to autonomously, safely, and smoothly execute natural language-driven navigation tasks is a core technical problem that urgently needs to be solved in the field of embodied intelligence and UAV navigation. Summary of the Invention

[0005] The purpose of this invention is to provide a semantic navigation method and system for unmanned aerial vehicles (UAVs) based on large models and reinforcement learning, aiming to solve the technical problems that existing UAV navigation technologies cannot effectively understand fuzzy natural language commands, and that large models are difficult to directly generate real-time, continuous low-level motion control signals.

[0006] The technical solution of this invention is: to provide a semantic navigation method for unmanned aerial vehicles (UAVs) based on large models and reinforcement learning, the method comprising:

[0007] S1. Acquire multimodal perception data of the environment surrounding the UAV, construct a three-dimensional metric semantic map containing object category and spatial location information, and extract environmental depth features;

[0008] S2. Input the natural language instructions and the semantic map into the first multimodal large model to extract or infer task-related semantic target information, wherein the semantic target information includes at least the target spatial location;

[0009] S3. Input the natural language instructions, historical navigation planning information, semantic target information, and UAV multi-directional environmental images into the second multimodal large model, and output the high-level motion direction;

[0010] S4. Generate a set of candidate navigation target points within the local feasible domain of the current location of the UAV. Calculate the comprehensive weight of each candidate navigation target point based on the historical trajectory distance, the high-level movement direction, the obstacle distance, and the target spatial location. Select the candidate navigation target point with the largest comprehensive weight as the local navigation target point.

[0011] S5. Input the local navigation target point, the current motion state information of the UAV and the environmental depth features into the pre-trained soft actor critic SAC reinforcement learning model, and output continuous speed control commands and heading angle control commands to drive the UAV to fly.

[0012] S6. Iteratively execute steps S1 to S5 above, and synchronously update the historical trajectory and historical navigation planning information of the UAV until the preset termination condition is met.

[0013] In any of the above technical solutions, further, step S2 extracts or infers task-related semantic target information, specifically including:

[0014] When a target object exists in the semantic map that directly matches a natural language instruction, the directly matched target object will be used as the final target output.

[0015] When there is no directly matching target object in the semantic map, the first multimodal big model infers an intermediate target object that has a semantic relationship with the natural language instruction based on common sense reasoning or prior knowledge of object placement.

[0016] In any of the above technical solutions, further, in step S3:

[0017] Multi-directional environmental images include real-time images from four directions: in front, left, right, and behind the drone.

[0018] The second multimodal large model comprehensively analyzes the obstacle distribution in real-time images from four directions, and combines semantic target information and historical navigation planning information to select the direction with the fewest obstacles and the most favorable direction for moving towards the semantic target as the high-level motion direction.

[0019] In any of the above technical solutions, further, in step S4, for any candidate navigation target point... Its overall weight It is calculated using the following formula:

[0020] ;

[0021] in, These are preset weighting coefficients; Distance weights are defined as follows: ,in For the historical trajectory of the drone, and For the historical trajectory Adjacent waypoints on the road It is the Euclidean distance function; Assuming the directional weights of the large model, let the angle between the relative vector of the candidate point and the direction of motion of the higher level be denoted as . ,when hour, ;when hour, ; The safety distance weight is defined as the candidate navigation target point. Distance field value in the Euclidean symbolic distance field ; Semantic weight is defined as follows: ,in For the target spatial location, This represents the Euclidean distance norm.

[0022] In any of the above technical solutions, further, in step S5:

[0023] The reward function of the SAC reinforcement learning model includes a single-step progress reward. The calculation formula is as follows:

[0024] ;

[0025] in, and These represent the Euclidean distances between the UAV and the local navigation target point at the current and previous moments, respectively. and These are the drone's heading angles at the current and previous moments, respectively. Forward velocity component, As the reward coefficient, The penalty constant;

[0026] The SAC reinforcement learning model employs an adaptive temperature parameter adjustment mechanism based on target distance. The update formula is:

[0027] ;

[0028] in, For learning rate, For about gradient operator, For state vectors, For action vectors, For parameters Actor network strategy, For distance The target entropy that decays.

[0029] In any of the above technical solutions, the method is further applied to at least one of the following scenarios:

[0030] Search and rescue missions, in which natural language instructions include descriptions of the people or objects to be searched;

[0031] Industrial inspection tasks, in which natural language commands include descriptions of the equipment to be inspected or anomalies;

[0032] Logistics transportation tasks, in which natural language instructions include descriptions of cargo placement or pickup points;

[0033] Security monitoring tasks, in which natural language instructions include descriptions of the area to be monitored or abnormal behavior.

[0034] A system for UAV semantic navigation based on large models and reinforcement learning, as described in any of the above technical solutions, is also provided. This system includes:

[0035] The perception and mapping module is used to acquire multimodal perception data of the environment around the UAV, construct a three-dimensional metric semantic map containing object category and spatial location information, and extract environmental depth features;

[0036] The sensing big model interface module is used to input natural language instructions and semantic maps into the first multimodal big model to extract or infer task-related semantic target information;

[0037] The planning large model interface module is used to input natural language commands, historical navigation planning information, semantic target information and UAV multi-directional environmental images into the second multimodal large model, and output the high-level motion direction;

[0038] The target calculation module is used to generate a set of candidate navigation target points within the local feasible domain of the current position of the UAV, calculate the comprehensive weight of each candidate navigation target point, and select the candidate navigation target point with the largest comprehensive weight as the local navigation target point;

[0039] The SAC path planning and control module is used to input local navigation target points, UAV current motion state information, and environmental depth features into the pre-trained SAC reinforcement learning model, and output continuous speed control commands and heading angle control commands.

[0040] In any of the above technical solutions, the sensing large model interface module and the planning large model interface module are further connected to the cloud server through the wireless communication module to perform large model inference in the cloud.

[0041] The perception and mapping module, target calculation module, and SAC path planning and control module are deployed on the UAV's onboard computer and run.

[0042] The beneficial effects of this invention are:

[0043] This invention enables UAVs to possess powerful natural language understanding and common sense reasoning capabilities through high-level collaboration between the first and second multimodal large models. Even if the target is not directly found in the current field of view or semantic map, it can autonomously infer reasonable intermediate exploration targets (such as actively inferring and heading to the shelf area when looking for a pipe), effectively solving the problem that traditional navigation cannot execute fuzzy commands.

[0044] This invention transforms the high-level directional commands output by the large model into quantifiable weights, and then weights and fuses them with historical trajectory distance, safe distance, and semantic distance to calculate the local navigation target point. This result is then fed into the SAC reinforcement learning model to generate continuous velocity and heading angle commands. This design bridges the gap between the high inference latency of large models and the high-frequency, real-time requirements of UAV low-level control, ensuring smooth flight trajectories while reducing collision rates through the obstacle avoidance reward mechanism of reinforcement learning.

[0045] In terms of system deployment, this invention offloads computationally intensive large model inference to the cloud, while the airborne terminal retains only lightweight perception mapping, target calculation, and reinforcement learning control modules. Without increasing the additional hardware computing power burden on the drone, it ensures the system's real-time response capability and the drone's endurance. Attached Figure Description

[0046] The advantages of the above and additional aspects of the present invention will become apparent and readily understood in the description of the embodiments in conjunction with the following drawings, wherein:

[0047] Figure 1 This is a schematic diagram of the overall framework of a UAV semantic navigation system based on large model and reinforcement learning according to an embodiment of the present invention;

[0048] Figure 2 This is a schematic diagram of the structured cue word design of the first multimodal large model (sensing LLM) of a UAV semantic navigation system based on large model and reinforcement learning according to an embodiment of the present invention;

[0049] Figure 3This is a schematic diagram of the structured cue word design for the second multimodal large model (planning MLM) of a UAV semantic navigation system based on large model and reinforcement learning according to an embodiment of the present invention;

[0050] Figure 4 This is a schematic diagram illustrating the dynamic navigation process of a UAV executing natural language commands in a simulation environment, based on a large model and reinforcement learning semantic navigation method and system according to an embodiment of the present invention.

[0051] Figure 5 This is a comparison chart of training curves for different depth reinforcement learning algorithms in a semantic navigation method and system for unmanned aerial vehicles based on large models and reinforcement learning according to an embodiment of the present invention.

[0052] Figure 6 The figure shows the experimental results of the impact of the planning distance parameter on the navigation performance of a UAV semantic navigation method and system based on large model and reinforcement learning according to an embodiment of the present invention.

[0053] Figure 7 This is a schematic diagram of the structural configuration of a real UAV hardware platform for a UAV semantic navigation system based on large model and reinforcement learning according to an embodiment of the present invention;

[0054] Figure 8 This is a schematic diagram illustrating the dynamic navigation process of a UAV executing natural language commands in a real-world environment, based on a large model and reinforcement learning semantic navigation method and system according to an embodiment of the present invention. Detailed Implementation

[0055] To better understand the above-mentioned objectives, features, and advantages of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that, unless otherwise specified, the embodiments of the present invention and the features thereof can be combined with each other.

[0056] In the following description, many specific details are set forth in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and therefore the scope of protection of the invention is not limited to the specific embodiments disclosed below.

[0057] like Figure 1 As shown, this embodiment provides a UAV semantic navigation system based on large-scale models and reinforcement learning. The system adopts a layered, decoupled, and weighted fusion architecture, comprising, from top to bottom, a semantic perception layer, a reasoning and decision-making layer, a target calculation layer, and a motion control layer. Specifically, the system includes: a perception and mapping module, a large-scale sensing model interface module, a large-scale planning model interface module, a historical information storage module, a target calculation module, and a soft path planning and control module.

[0058] In terms of hardware deployment and computing power allocation, such as Figure 7 As shown, the real drone hardware platform of the present invention includes a hexacopter drone frame, and the main body of the fuselage is equipped with the following components: a GNSS module for providing centimeter-level positioning; a depth camera for acquiring RGB images and depth images; an inertial measurement unit for providing three-axis acceleration and angular velocity data; a flight controller for low-level attitude and motor control; a 4G / 5G or Wi-Fi wireless communication module for communicating with a cloud server; and an onboard computer.

[0059] To optimize computational load, this system offloads the computationally intensive large model inference portion to the cloud via API interfaces. Specifically, the sensing large model interface module and the planning large model interface module run in the cloud; while the UAV's onboard computer only handles the lightweight perception and mapping module, target calculation module, and SAC path planning and control module. The sensing and planning large model interface modules invoke the large model less frequently than the control command generation frequency of the SAC path planning and control module, thus improving endurance while ensuring system real-time performance.

[0060] Based on the above system architecture, this invention provides a UAV semantic navigation method based on a large model and reinforcement learning, specifically including the following steps:

[0061] S1. Acquire multimodal perception data of the environment surrounding the drone in real time, and construct a three-dimensional metric semantic map containing object category and spatial location information based on the multimodal perception data.

[0062] Specifically, the perception and mapping module mounted on the UAV acquires RGB images, depth images, and IMU inertial data of the environment in real time. The system uses an online semantic SLAM pipeline to process the above data: real-time semantic segmentation is performed using a semantic segmentation network, pose estimation is performed using visual inertial odometry, and a 3D metric semantic map is generated in real time using a mapping algorithm. The map is stored in an octree or similar voxel data structure, with each voxel containing spatial coordinates and semantic labels.

[0063] Simultaneously, the system generates a Euclidean symbolic distance field based on depth data to represent the distance from each spatial point in the environment to the nearest obstacle. Furthermore, the raw depth image is input into a pre-trained variational autoencoder, compressed into a low-dimensional latent feature vector, and used as the environment depth feature encoding. This is for use by the reinforcement learning module in the future.

[0064] S2. Input the natural language instructions input by the user and the semantic map at the current moment into the first multimodal large model to extract or infer the semantic targets related to the task.

[0065] like Figure 2 As shown, the sensor large model interface module will process the natural language commands input by the user. Semantic map at the current moment The historical target objects and historical target arrival markers in the historical information storage module are input into the first multimodal large model deployed in the cloud.

[0066] The system guides the model in semantic parsing using structured cue words containing four fixed parts: task description, input information, selection strategy, and output format. Its mapping function can be expressed as: .

[0067] When semantic map There are natural language instructions in it When the target object is directly matched, the directly matched target object is output as the final target; when the semantic map... When there is no directly matching target object, the first multimodal big model infers an intermediate target object that is semantically related to the instruction from the semantic map based on common sense reasoning or prior knowledge of object placement.

[0068] This step ultimately outputs structured semantic information, which includes at least: target object labels. Target spatial location And a flag indicating whether the current object is the final target. (Boolean value).

[0069] S3. Input natural language commands, historical navigation planning information, semantic target information, and multi-directional environmental images into the second multimodal large model, and output structured directional commands.

[0070] like Figure 3 As shown, the planning large model interface module will use natural language commands. Historical navigation planning information (Including the historical movement trajectory completed by the UAV and the historical direction command sequence executed), semantic target information output in step S2 (Includes target tags) and its location (and real-time environmental images of the drone in the four directions of front, left, right, and rear). They are jointly input into the second multimodal large model (i.e. planning MLM) deployed in the cloud.

[0071] Its mapping function can be expressed as: .

[0072] This model comprehensively analyzes the obstacle distribution in images from four directions, combines it with semantic target information, and selects the direction with the fewest obstacles and the most favorable direction for moving towards the semantic target. The final output is a structured direction command, including at least one high-level motion direction. and its corresponding reasoning explanation .

[0073] S4. Generate a set of candidate navigation target points within the local feasible region of the UAV's current location, calculate the comprehensive weight, and select the local navigation target points.

[0074] The target calculation module uses the current position of the UAV Centered on, within a preset radius and height Within the cylindrical space (i.e., the local feasible region), discrete candidate navigation target points are generated at preset sampling intervals. For each candidate target point Calculate its overall weight This weight is obtained by weighted summation of the following four factors (distance, large model orientation, safety distance, and semantic weight):

[0075] Distance weight This characterizes the shortest distance between a candidate target point and the historical flight path of a UAV, encouraging the exploration of new areas. A candidate point is defined as... To the historical trajectory of the drone The shortest Euclidean distance of the resulting polyline segment, i.e. .

[0076] Large model directional weights : Used to characterize the direction of the candidate target point relative to the high-level motion direction output in step S3 The angular consistency between them. Let the relative vectors of the candidate points be... With direction The included angle between them is .when When, the weight is defined as ;when When the weight is 0, the weight is 0.

[0077] Safety distance weight This is used to characterize the distance from a candidate target point to the nearest obstacle, ensuring flight safety. Candidate target points are used directly. The Euclidean sign distance field value at that location, i.e. .

[0078] Semantic weight : Used to characterize the spatial location of candidate target points relative to the target output in step S2. The distance between them guides the drone closer to the target. This is defined as a negative exponential decay function, i.e. .

[0079] Finally, the formula for calculating the overall weight is:

[0080] ;

[0081] in, to These are preset weighting coefficients. The system selects the candidate target point with the highest overall weight. As the local navigation target point at the current moment.

[0082] S5. Encode the local navigation target point, motion state information and environmental depth features to form a state space, input it into the SAC reinforcement learning model to output control commands.

[0083] The SAC path planning and control module will output the local navigation target point from step S4. The input is transformed into the UAV's state space, and a pre-trained SAC reinforcement learning model outputs continuous low-level control commands. The specific mechanism is as follows:

[0084] Define the state space: Time, State Space .in, The current speed of the drone. From the current location of the drone to the local navigation target point The relative position vector, The current heading angle of the drone. This is the encoding of the environmental depth features after dimensionality reduction via VAE encoding in step S1.

[0085] Define the action space: The SAC model outputs a continuous action space. That is, continuous speed control commands (speed increments) ) and heading angle control commands (heading angle increments) The command is sent to the flight controller to drive the drone to fly toward the local navigation target point.

[0086] Design the reward function: To guide the drone smoothly and safely towards the target, the single-step progress reward is defined as:

[0087] ;

[0088] in, and These represent the Euclidean distances between the UAV and the local navigation target point at the current and previous moments, respectively. and These are the heading angles at the current time and the previous time, respectively. Forward velocity component, These are the weighting coefficients. A penalty constant to prevent drone stagnation. The complete reward function. A hierarchical structure is adopted: a significant reward is given when the drone reaches the target point. A severe collision penalty is imposed upon a collision. Penalties will be imposed for flying out of the boundary or exceeding the time limit. and The aforementioned progress bonus will be awarded during normal flight operations. .

[0089] Adaptive temperature parameter adjustment mechanism: temperature parameter The update formula is:

[0090] ;

[0091] in, For learning rate, For about gradient operator, For state vectors, For action vectors, For parameters Actor network strategy, For distance The target entropy that decays.

[0092] S6. The system iteratively executes steps S1 to S5 above. As the drone moves, the semantic map is continuously updated. This continues until step S2 outputs a flag indicating whether the target is the final objective. It is true, and the drone has reached the target spatial location. The navigation task ends when a preset termination condition such as a collision or timeout is triggered.

[0093] The method provided by this invention can be widely applied to a variety of complex scenarios, including but not limited to: search and rescue missions, wherein the natural language instructions include a description of the person or object to be searched; industrial inspection missions, wherein the natural language instructions include a description of the equipment to be inspected or anomalies; logistics and transportation missions, wherein the natural language instructions include a description of the cargo placement point or pickup point; and security monitoring missions, wherein the natural language instructions include a description of the area to be monitored or abnormal behavior.

[0094] Figure 5 This is a comparison of training curves for different deep reinforcement learning algorithms in this embodiment of the invention. The horizontal axis represents the number of training episodes, ranging from 0 to 1000. The vertical axis represents the average cumulative reward per episode, ranging approximately from 200 to +100.

[0095] The figure shows four curves:

[0096] DDPG (dark blue dotted line): The initial reward is about 150, which slowly increases, reaching about 50 after about 600 rounds, after which it fluctuates greatly and converges unstablely.

[0097] PPO (orange dotted line): The initial reward is about 120, and the rate of increase is better than DDPG. It reaches about 0 after about 500 rounds, but the growth slows down thereafter and eventually stabilizes at about +20.

[0098] SACw / oVAE (green dotted line, i.e., the SAC of this invention but without using VAE to encode the depth map, directly processed by CNN): The initial reward is about 100, the convergence speed is relatively fast, reaching +30 after about 400 rounds, but then there are slight fluctuations.

[0099] SACwithVAE (red solid line, complete configuration of this invention): The initial reward is about 80, the convergence speed is the fastest, reaching +40 after about 300 rounds, and continues to rise steadily, finally reaching about +70 after 800 rounds, which is significantly higher than other algorithms.

[0100] pass Figure 5 It can be proven that the SAC algorithm outperforms DDPG and PPO in UAV navigation tasks; using VAE to encode and compress the depth map can accelerate convergence and improve training stability.

[0101] To further illustrate the working process and beneficial effects of the present invention in practical applications, three specific embodiments are listed below.

[0102] Example 1: Navigation method in a simulation environment.

[0103] This embodiment is implemented in the AirSim high-fidelity simulation platform. For example... Figure 4 As shown, the simulation scene is a factory-style indoor map containing shelves, tables and chairs, corridors, and randomly placed obstacles.

[0104] During system initialization, the UAV's starting position is set to (0,0,1.5)m in the world coordinate system, with an initial heading angle of 0°. The user inputs the natural language command "Findsomepipes" (find some pipes) through the ground station software. At this point, the UAV's semantic map is empty.

[0105] After the drone takes off, the onboard synchronous RGBD camera begins to collect data. The RGB images are input into the Enet-based semantic segmentation network, the depth images are used for odometry estimation, and a 3D metric semantic map is generated in real time through the Kimera-Semantics online SLAM pipeline.

[0106] Subsequently, the system calls the API of the first multimodal large model (Sensing LLM, DeepSeekV3). The Sensing LLM parsing instructions identifies the key target as "pipe," but after traversing the current semantic map, no "pipe" or synonym tags are found. At this point, the Sensing LLM triggers common-sense reasoning: based on prior knowledge of the indoor environment, pipes are usually stored on shelves or storage racks. Therefore, the Sensing LLM locates a "shelf" type object from the semantic map, extracts its center position (5.2, 3.1, 0.8) m, and outputs structured information: <reason>No pipe was found; based on common sense, it is inferred that the pipe may be on the shelf.<Target_Object> :shelf;<Target_Position> :(5.2,3.1,0.8);<Whether_final_target> :False".

[0107] Next, the system calls the API of the second multimodal large model (Planning MLM, GPT4o). Planning MLM analyzes the images from four directions and finds: the foreground image shows a relatively open passageway but no shelves visible in the distance; the left image shows an open area with vaguely visible shelf-like structures; the right image is adjacent to a wall with no passageway; and the background is the starting area that has already been flown over. After comprehensive judgment, Planning MLM outputs: " <reason>The left side is open and aligned with the direction of the shelves, while the right side is a wall; <direction>Move to the left front.

[0108] During the target calculation phase, the target calculation module generates discrete candidate points with a sampling interval of 0.5m within a cylindrical space centered on the UAV's current location, with a radius of 10m and a height of 2m. The system calculates a comprehensive weight for each candidate point; in this embodiment, the weight coefficient is set to... (Distance weight) (Large model directional weights) (Safety distance weight) (Semantic weight). After weighted calculation, a candidate point located about 3.5m to the left front of the UAV received the highest comprehensive weight and was selected as the local navigation target point.

[0109] In the low-level control phase, the SAC module receives the relative position of the local navigation target point, the UAV's current velocity (0,0,0) m / s, the current heading angle (0°), and a 64-dimensional depth feature vector encoded by VAE. The pre-trained SAC model outputs continuous actions: a velocity increment of 0.25 m / s (forward) and a heading angle increment of 12° / s (left turn). This action command is sent to the flight controller, driving the UAV to smoothly turn and accelerate towards the target point.

[0110] After flying for approximately 15 seconds, the drone reached the vicinity of the shelf. At this point, a new "pipe" label was added to the updated semantic map, located at (6.8, 2.9, 0.9) m. The sensor LLM detected this change, immediately updated the target, and output: <reason>Pipeline detected, updated to final target;<Target_Object> :pipe;<Target_Position> :(6.8,2.9,0.9);<Whether_final_target> ":True". Based on the new semantic target and visual image, the planning MLM outputs "Move to the right and forward". In subsequent iterations, the SAC model guides the drone to successfully reach the pipe position, and the mission is completed.

[0111] Under the above simulation settings, 100 test rounds were conducted with random starting positions and obstacle layouts. Statistical results show that the system of this invention achieves the following: average trajectory length of 20.82m, navigation error of 2.87m, task success rate of 75.93%, collision rate of only 16.67%, and path weighted success rate (SPL) of 58.78%. All indicators are significantly better than existing end-to-end reinforcement learning navigation methods.

[0112] Example 2: Real-world experimental verification.

[0113] This embodiment verifies the effectiveness of the invention in a real physical environment. For example... Figure 7 As shown, the real-world drone hardware platform uses a hexacopter frame with a 450mm wheelbase, equipped with a UM982 GNSS module, an Intel RealSense D435 depth camera, an IMU, an NVIDIA Orin NX onboard computer, a 4G / 5G wireless communication module, and a Pixhawk flight controller. The experiment was conducted in a real office environment of approximately 15m × 10m, including objects such as a desk, chair, monitor, filing cabinet, and headphones. No pre-built map was used; the drone took off from the corridor entrance.

[0114] The user enters the command: "Please find headphones". For example... Figure 8 As shown, the execution process is divided into the following stages:

[0115] In Phase A (Initial Exploration), the drone is located at the entrance of the corridor, with multiple office cubicles visible in its field of view. Sensor LLM examines the semantic map but finds no "headphones" tag and, based on common sense, does not output a valid intermediate target. Planning LLM analyzes the four-directional image and identifies multiple desks in the right-hand office area, determining it as a promising search direction, and outputs "Move right".

[0116] In Phase B (Continuous Exploration), the drone flies approximately 5 meters to the right before turning right into an office cubicle. Two desks appear in the field of view, each with a monitor and keyboard. The semantic map updates to show the labels "desk" and "monitor," but headphones are still not detected. The planning MLM infers that the desks ahead may contain a target and outputs "Move forward," guiding the drone to continue exploring deeper.

[0117] In Phase C (Target Discovery and Navigation), when the drone moves to a position approximately 1.5m above a desk, the semantic segmentation network successfully identifies a pair of black headphones on the desk, with location coordinates (4.3, 0.5, 1.1)m. The sensor-based LLM immediately updates the target to "headphones" and marks it as the final target. The planning LLM cross-validates the visual input, confirming that the headphones are located in the left field of view, and outputs "Move left".

[0118] In Phase D (reaching the target), the SAC controller generates a smooth left turn and descent trajectory based on the updated local target point. Finally, the drone adjusts its position and hovers precisely about 0.5m directly above the headset, and the system prompts that the mission is complete.

[0119] Twenty repeated flight tests were conducted in the same scenario, and the quantitative results are as follows: the mission success rate was as high as 90% (18 / 20), the collision rate was only 5% (1 / 20), the inference latency of the large model and SAC policy was 1820±142ms, the end-to-end latency was 2152±181ms, the average trajectory length was 23.4±6.1m, and the path-weighted success rate was 0.62±0.18. These results fully verify the robustness, safety, and real-time feasibility of this invention under real-world lighting, texture, and object shape changes.

[0120] Example 3: Parameter analysis and ablation experiment.

[0121] To further verify the effectiveness of the key parameters and modules in this invention, the following parameter analysis and ablation experiments were conducted.

[0122] The first step is an impact analysis of the planning distance parameters. For example... Figure 6 As shown, the experiment tested the impact of the local feasible region radius on navigation performance in step S4. The results show that when... When the setting is too small (e.g., 5m), the drone needs to frequently replan local target points, resulting in fragmented flight trajectories, increased total path length, and low efficiency; when When the distance is set too large (e.g., 15m or more), candidate target points are very likely to fall into unknown areas that have not yet been mapped or have a high density of obstacles, leading to a sharp increase in collision risk. Experiments have shown that the optimal planning distance is 10m, which perfectly matches the effective sensing range of the UAV's onboard depth camera. At this distance, the success rate is the highest, achieving the best balance between exploration efficiency and flight safety.

[0123] Secondly, an ablation experiment was conducted to assess the importance of historical information. This invention introduces a historical information storage module in steps S2 and S3. The ablation experiment results show that removing the historical planned trajectory (PP) causes the UAV to lose its temporal continuity memory, frequently re-exploring already visited areas, significantly increasing path length and decreasing success rate. Removing the historical semantic target (PT) disrupts the logical coherence of the navigation task, making the UAV prone to getting lost in complex environments. When both are removed simultaneously, the system performance deteriorates most significantly, with the path-weighted success rate (SPL) plummeting from a full 58.78% to 21.83%. This fully demonstrates the indispensable and crucial role of the historical information storage module in maintaining navigation consistency and improving search efficiency in this invention.

[0124] In summary, this invention proposes a semantic navigation method for unmanned aerial vehicles (UAVs) based on large models and reinforcement learning, comprising:

[0125] S1. Acquire multimodal perception data of the environment surrounding the drone, construct a three-dimensional metric semantic map containing object category and spatial location information, and extract environmental depth features.

[0126] S2. Input the natural language instructions and the semantic map into the first multimodal large model to extract or infer the semantic target information related to the task. The semantic target information includes at least the target spatial location.

[0127] S3. Input the natural language instructions, historical navigation planning information, semantic target information and UAV multi-directional environmental images into the second multimodal large model, and output the high-level motion direction.

[0128] S4. Generate a set of candidate navigation target points within the local feasible domain of the current location of the UAV. Calculate the comprehensive weight of each candidate navigation target point based on the historical trajectory distance, the high-altitude movement direction, the obstacle distance, and the target spatial position. Select the candidate navigation target point with the largest comprehensive weight as the local navigation target point.

[0129] S5. Input the local navigation target point, the current motion state information of the UAV, and the environmental depth features into the pre-trained soft actor critic SAC reinforcement learning model, and output continuous speed control commands and heading angle control commands to drive the UAV to fly.

[0130] S6. Iteratively execute steps S1 to S5 above, and synchronously update the historical trajectory and historical navigation planning information of the UAV until the preset termination condition is met.

[0131] A system applying the aforementioned UAV semantic navigation method based on large models and reinforcement learning is also provided, including:

[0132] The perception and mapping module is used to acquire multimodal perception data of the environment around the UAV, construct a three-dimensional metric semantic map containing object category and spatial location information, and extract environmental depth features.

[0133] The sensing large model interface module is used to input natural language instructions and semantic maps into the first multimodal large model to extract or infer task-related semantic target information.

[0134] The planning large model interface module is used to input natural language commands, historical navigation planning information, semantic target information, and UAV multi-directional environmental images into the second multimodal large model, and output the high-level motion direction.

[0135] The target calculation module is used to generate a set of candidate navigation target points within the local feasible domain of the UAV's current location, calculate the comprehensive weight of each candidate navigation target point, and select the candidate navigation target point with the largest comprehensive weight as the local navigation target point.

[0136] The SAC path planning and control module is used to input local navigation target points, UAV current motion state information, and environmental depth features into the pre-trained SAC reinforcement learning model, and output continuous speed control commands and heading angle control commands.

[0137] The steps in this invention can be adjusted, combined, or deleted according to actual needs.

[0138] The units in the device of the present invention can be merged, divided, or reduced according to actual needs.

[0139] In this invention, the terms "installation," "connection," "linking," and "fixing" should be interpreted broadly. For example, "connection" can be a fixed connection, a detachable connection, or an integral connection; "linking" can be a direct connection or an indirect connection through an intermediate medium. Those skilled in the art can understand the specific meaning of these terms in this invention according to the specific circumstances.

[0140] The shapes of the components in the accompanying drawings are schematic and may differ from their actual shapes. The drawings are only used to illustrate the principles of the present invention and are not intended to limit the present invention.

[0141] Although the invention has been disclosed in detail with reference to the accompanying drawings, it should be understood that these descriptions are merely exemplary and not intended to limit the application of the invention. The scope of protection of the invention is defined by the appended claims and may include various variations, modifications, and equivalents made to the invention without departing from the scope and spirit of the invention.< / reason> < / direction> < / reason> < / reason>

Claims

1. A semantic navigation method for unmanned aerial vehicles (UAVs) based on large models and reinforcement learning, characterized in that, The method includes: S1. Acquire multimodal perception data of the environment surrounding the drone, construct a three-dimensional metric semantic map containing object category and spatial location information, and extract environmental depth features; S2. Input the natural language instructions and the semantic map into the first multimodal large model to extract or infer task-related semantic target information, wherein the semantic target information includes at least the target spatial location; S3. Input the natural language instructions, historical navigation planning information, semantic target information, and UAV multi-directional environmental images into the second multimodal large model, and output the high-level motion direction; S4. Generate a set of candidate navigation target points within the local feasible domain of the current location of the UAV. Calculate the comprehensive weight of each candidate navigation target point based on the historical trajectory distance, the high-level movement direction, the obstacle distance, and the target spatial location. Select the candidate navigation target point with the largest comprehensive weight as the local navigation target point. S5. Input the local navigation target point, the current motion state information of the UAV and the environmental depth features into the pre-trained soft actor critic SAC reinforcement learning model, and output continuous speed control commands and heading angle control commands to drive the UAV to fly. S6. Iteratively execute steps S1 to S5 above, and synchronously update the historical trajectory and historical navigation planning information of the UAV until the preset termination condition is met.

2. The UAV semantic navigation method based on large model and reinforcement learning as described in claim 1, characterized in that, Step S2 involves extracting or inferring task-related semantic target information, specifically including: When a target object exists in the semantic map that directly matches a natural language instruction, the directly matched target object will be used as the final target output. When there is no directly matching target object in the semantic map, the first multimodal big model infers an intermediate target object that has a semantic relationship with the natural language instruction based on common sense reasoning or prior knowledge of object placement.

3. The UAV semantic navigation method based on large model and reinforcement learning as described in claim 1, characterized in that, In step S3: Multi-directional environmental images include real-time images from four directions: in front, left, right, and behind the drone. The second multimodal large model comprehensively analyzes the obstacle distribution in real-time images from four directions, and combines semantic target information and historical navigation planning information to select the direction with the fewest obstacles and the most favorable direction for moving towards the semantic target as the high-level motion direction.

4. The UAV semantic navigation method based on large model and reinforcement learning as described in claim 1, characterized in that, In step S4, for any candidate navigation target point Its overall weight It is calculated using the following formula: ； in, These are preset weighting coefficients; Distance weights are defined as follows: ,in For the historical trajectory of the drone, and For the historical trajectory Adjacent waypoints on the road It is the Euclidean distance function; Assuming the directional weights of the large model, let the angle between the relative vector of the candidate point and the direction of motion of the higher level be denoted as . ,when hour, ;when hour, ; The safety distance weight is defined as the candidate navigation target point. Distance field value in the Euclidean symbolic distance field ; Semantic weight is defined as follows: ,in For the target spatial location, This represents the Euclidean distance norm.

5. The UAV semantic navigation method based on large model and reinforcement learning as described in claim 1, characterized in that, In step S5: The reward function of the SAC reinforcement learning model includes a single-step progress reward. The calculation formula is as follows: ； in, and These represent the Euclidean distances between the UAV and the local navigation target point at the current and previous moments, respectively. and These are the drone's heading angles at the current and previous moments, respectively. Forward velocity component, As the reward coefficient, The penalty constant; The SAC reinforcement learning model employs an adaptive temperature parameter adjustment mechanism based on target distance. The update formula is: ； in, For learning rate, For about gradient operator, For state vectors, For action vectors, For parameters Actor network strategy, For distance The target entropy that decays.

6. The UAV semantic navigation method based on large model and reinforcement learning as described in claim 1, characterized in that, The method is applicable to at least one of the following scenarios: Search and rescue missions, in which natural language instructions include descriptions of the people or objects to be searched; Industrial inspection tasks, in which natural language commands include descriptions of the equipment to be inspected or anomalies; Logistics transportation tasks, in which natural language instructions include descriptions of cargo placement or pickup points; Security monitoring tasks, in which natural language instructions include descriptions of the area to be monitored or abnormal behavior.

7. A system applying the UAV semantic navigation method based on large model and reinforcement learning as described in any one of claims 1 to 6, characterized in that, The system includes: The perception and mapping module is used to acquire multimodal perception data of the environment around the UAV, construct a three-dimensional metric semantic map containing object category and spatial location information, and extract environmental depth features; The sensing big model interface module is used to input natural language instructions and semantic maps into the first multimodal big model to extract or infer task-related semantic target information; The planning large model interface module is used to input natural language commands, historical navigation planning information, semantic target information and UAV multi-directional environmental images into the second multimodal large model, and output the high-level motion direction; The target calculation module is used to generate a set of candidate navigation target points within the local feasible domain of the current position of the UAV, calculate the comprehensive weight of each candidate navigation target point, and select the candidate navigation target point with the largest comprehensive weight as the local navigation target point; The SAC path planning and control module is used to input local navigation target points, UAV current motion state information, and environmental depth features into the pre-trained SAC reinforcement learning model, and output continuous speed control commands and heading angle control commands.

8. The UAV semantic navigation system based on large model and reinforcement learning as described in claim 7, characterized in that, The sensing large model interface module and the planning large model interface module are connected to the cloud server via a wireless communication module to perform large model inference in the cloud. The perception and mapping module, target calculation module, and SAC path planning and control module are deployed on the UAV's onboard computer and run.

Images