An indoor unmanned aerial vehicle hierarchical path planning method and system based on a three-dimensional Gaussian splash and point cloud combined model

Abstract

This invention discloses a hierarchical path planning method and system for indoor unmanned aerial vehicles (UAVs) based on a joint model of 3D Gaussian splash and point cloud, relating to the field of indoor UAV technology. This invention constructs a joint map of a laser point cloud model and a 3D Gaussian splash visual model, balancing high-precision geometric collision detection and high-fidelity visual information to achieve task-driven UAV path planning. It employs a high-altitude-priority hierarchical global planning strategy, dividing the indoor airspace into multiple altitude layers and prioritizing the use of high-altitude main channels for rapid coverage flight, reducing global flight time. Upon approaching target markers, an incremental update mechanism of the local 3D Gaussian splash model is triggered to optimize scene representation in real time and generate a refined shooting path. This invention also provides corresponding system modules, including a data acquisition and mapping module, and a hierarchical path planning module. This invention enables efficient, safe, and high-precision autonomous navigation and mission execution for indoor UAVs in complex environments.

Description

Technical Field

[0001] This invention relates to the field of indoor drone technology, specifically a hierarchical path planning method and system for indoor drones based on a joint model of three-dimensional Gaussian splashing and point cloud. Background Technology

[0002] With the rapid development of drone technology, indoor drones are increasingly being used in building inspection, equipment monitoring, indoor surveying, and emergency search and rescue. Unlike outdoor flights, indoor drones face multiple challenges, such as limited space, dense obstacles, and lack of GPS signals, which places higher demands on their autonomous path planning capabilities.

[0003] Currently, indoor drone path planning mainly relies on LiDAR, visual sensors, or a fusion of both to construct an environmental map, and then performs path search and planning based on the map. Existing technical solutions can be divided into the following categories: (1) Map construction and path planning based on laser point clouds: The lidar acquires three-dimensional point cloud data of the environment through scanning, and uses occupancy grid maps, octree maps or voxel maps to represent obstacles, and then uses A Path planning is performed using algorithms such as the Fast Random Search Tree (RRT) or Dynamic Window (DWA). These methods have high accuracy in distance measurement, but they are not suitable for slender objects (such as wires, ropes, etc.) where reflection points are sparse, resulting in a large number of areas with missing information in the scene. Furthermore, laser point clouds lack texture and color information, making it difficult to support task-driven fine-grained path planning (such as shooting markers with specific color markings or texture features).

[0004] (2) Visual SLAM-based map building and navigation: Using an RGB-D camera or a monocular / binocular camera, sparse or semi-dense point cloud maps of the environment are built using algorithms such as ORB-SLAM and VINS, and then combined with depth images for obstacle avoidance and path planning. This type of method can obtain rich texture and color information, which is beneficial for performing vision-based tasks (such as marker recognition and photography). However, pure vision methods are prone to failure in areas with weak texture, and the sparse point cloud maps built by traditional visual SLAM are difficult to provide high-precision geometric information, often resulting in flying close to obstacles when planning paths, making it difficult to guarantee safety.

[0005] (3) Recent Advances of 3D Gaussian Splatting (3DGS) in Robot Navigation: In recent years, 3D Gaussian splatting, as an emerging 3D scene representation technology, has attracted widespread attention due to its high-fidelity rendering quality and real-time performance. Splat-Nav proposed the first robot navigation pipeline based on 3DGS scene representation, but this method is mainly aimed at ground robots and focuses on navigation and localization functions under a single representation. GRAD-NAV combines 3DGS with differentiable reinforcement learning for UAV navigation policy training. The above methods have made some progress in small-scale indoor scenes, but still have the following shortcomings: First, the 3DGS map itself does not contain explicit geometric collision information, and navigation safety depends on additional collision detection mechanisms; second, the 3DGS map building process has a large computational overhead, making it difficult to achieve real-time coverage in large-scale indoor scenes.

[0006] (4) Limitations of existing industrial applications: Existing patents focus on the modeling method itself, without addressing path planning or improving the visual effects of 3D reconstruction, and without solving the task-driven path planning problem.

[0007] In summary, existing indoor UAV path planning methods share the following common problems: (1) A single representation method is difficult to balance geometric accuracy and environmental semantic information; (2) There is a lack of efficient joint representation and rapid update mechanisms in large-scale indoor scenes; (3) Path planning is mostly global static planning, which is difficult to adapt to the dynamic changes in the local environment in tasks such as indoor marker shooting; (4) Failed to design differentiated path planning strategies based on task characteristics.

[0008] Therefore, there is an urgent need for an indoor UAV path planning method that integrates multimodal environment representation, supports task-driven hierarchical planning, and balances global efficiency with local accuracy. Summary of the Invention

[0009] The purpose of this invention is to provide an indoor UAV hierarchical path planning method and system based on a three-dimensional Gaussian splash and point cloud joint model, so as to solve the problems mentioned in the background art.

[0010] To achieve the above objectives, the present invention provides the following technical solution: a hierarchical path planning method for indoor UAVs based on a joint model of 3D Gaussian splash and point cloud, comprising the following steps: S1: Dual-mode joint map pre-construction: Control the UAV to traverse the target area, collect point cloud data using LiDAR, and collect RGB and depth images using an RGB-D camera; filter the point cloud data to construct a point cloud model, and use voxelization to construct an occupied grid map; use multi-view images and depth images collected by the RGB-D camera to construct an initial 3DGS model of the indoor scene through a 3D Gaussian splash reconstruction process; establish a spatial mapping index between the point cloud model and the 3DGS model; S2: Global path planning based on high-altitude priority layering strategy: Divide the flight airspace vertically into L altitude layers; set the high-altitude layer as the main channel layer for global path planning; receive task instructions containing the location and attributes of target markers, and generate a high-altitude priority global path from the starting point to the target markers; S3: Real-time local scene update based on 3D Gaussian splashing: When the horizontal projection distance between the drone and the target marker is less than or equal to a preset threshold, a local update area is defined with the target marker as the center, and the 3DGS model is incrementally updated in this area. S4: Local fine path planning: After the marker pose recognition is completed and the local 3DGS model is updated, the optimal path from the current position of the UAV to the optimal shooting position of the marker is generated and executed.

[0011] As a preferred embodiment of the present invention, step S1 specifically includes: S1.1 Data Acquisition and Preprocessing: Control the UAV to traverse the target indoor area along a preset scanning path, using the onboard LiDAR to collect point cloud data of the environment, and simultaneously using an RGB-D camera to collect RGB images and depth images from various viewpoints. The point cloud data is denoted as... Each point contains three-dimensional coordinates. and possible reflection intensity information; the RGB image sequence is denoted as The corresponding depth image is denoted as The RGB-D camera simultaneously records the camera pose during acquisition; S1.2 Constructing a point cloud model: The original point cloud collected by the lidar is filtered to obtain a dense point cloud model. On this point cloud model, an occupied grid map is constructed using the voxelization method, where 0 represents free space and 1 represents complete occupation by obstacles. The point cloud model provides high-precision geometric collision detection information for the UAV and is used for safe distance verification in global path planning. S1.3 Constructing the initial 3D Gaussian splash model: Using the multi-view image sequence and corresponding depth image acquired by the RGB-D camera, the initial 3DGS model of the indoor scene is constructed through the 3DGS reconstruction process. After densification and pruning optimization operations, a high-fidelity visual model of the indoor scene is obtained, which provides texture and color information for the drone to support the identification and tracking of markers. S1.4 Establish a dual-model joint map index: Establish a spatial mapping index between the point cloud model PointMap and the 3D Gaussian splash model 3DGSMap to ensure that in the world coordinate system... any spatial location The corresponding voxel occupancy information and 3D Gaussian point information can be quickly found.

[0012] As a further preferred embodiment of the present invention: step S2 specifically includes: S2.1 Establishing a spatial height hierarchy model: Based on the actual structure of the indoor environment and the mission requirements of the UAV, the flight airspace is divided into L height layers along the vertical direction, denoted as LL. ,in The highest level The lowest layer is defined by the point cloud distribution density of the indoor environment. The height layers are adaptively divided based on the density of the point cloud distribution in the indoor environment. The height layers are increased in areas with dense obstacles and decreased in areas with open space. S2.2 Setting the High-Altitude Priority Planning Principle: The high-altitude layer is designated as the main channel layer for global path planning, denoted as... In global path planning, priority is given to guiding the drone to... Conduct long-range coverage flights; S2.3 Generates a high-altitude priority global path: Receives task instructions from the user, which include a set of one or more target marker locations. And the attributes of the marker, for each target marker Perform the following steps: S2.3.1 Altitude Conversion Between Start and End Points: Assume the UAV is currently in the world coordinate system. The position in the middle is The target waypoint is .like Located in the low-altitude region and If the distance is too far, a layer from the current altitude to the main high-altitude passage will be generated. Height adjustment path This path prioritizes vertical ascent to avoid long horizontal crossings; S2.3.2 Main Channel Layer Path Planning: In the context of... On a plane centered on the height, using the occupancy information planned by the point cloud model, A The algorithm generates an unobstructed global path from the starting projection point to the target projection point. During path search, the cost function for expanding nodes is defined as: in, Let be the actual flight cost from the starting point to node n; h(n) is the heuristically estimated cost from node n to the target point. The cost for safe separation is based on the distance between node n and the nearest voxel obstacle. Calculation, defined as The aim is to guide the path as far away from obstacles as possible; These are adjustable weighting coefficients; S2.3.3 Terminal Altitude Descent Path Planning: This involves the UAV flying towards the target marker. When nearby, The horizontal projection distance from the target location is less than a threshold. Generate from the high-altitude main channel layer Sink to the height of the target marker descent path The descent path is executed primarily along the vertical direction, and rapid collision detection is performed on the voxels along the path. S2.3.4 Global Path Composition: and The three paths are pieced together in sequence to form... arrive Full global path For multiple target markers, the Traveling Salesman Problem optimization algorithm is used to determine the optimal access order, and then the global path sequence is synthesized sequentially. S3: Real-time local scene updates based on 3D Gaussian splashing When the drone flies to the vicinity of the target marker, that is, at close range to the target marker The horizontal projection distance is less than A local scene real-time update mechanism is initiated to incrementally update the 3DGS model for the local area around the target marker; S3.1 Detection trigger condition: The system monitors the drone in real time. With the current target marker Horizontal projection distance between ,when At that time, a partial scene update process is triggered. These parameters are adaptively set based on the camera's maximum effective sensing distance and are dynamically adjusted through online calibration. S3.2 Local area demarcation: using target markers Set the safe distance radius for drones at the center. The local safety buffer radius of the drone And delineate local update areas. The following conditions must be met: S3.3 Incremental 3D Gaussian splash mapping: for local areas The specific sub-steps for remodeling or optimization updates include: S3.3.1 Data Acquisition and Pose Optimization: The UAV hovers or performs minor maneuvers around the target marker, continuously acquiring RGB-D image sequences. And under the guidance of the joint map index, the camera pose estimation is optimized in real time. The 3DGS model is used to render the 3D Gaussian point set in the current area, and the rendered image is matched with the real image to optimize the pose. S3.3.2 Incremental addition of 3D Gaussian points: Identify local regions based on newly acquired image data. For regions that are not yet covered by the existing 3DGS model or whose reconstruction quality is insufficient, 3D Gaussian point clouds are initialized and inserted into the 3DGS model. The parameters of the inserted 3D Gaussian points are optimized by minimizing the loss function between the rendered image and the real image. S3.3.3 Parameter optimization and pruning of 3D Gaussian points: Perform several rounds of iterative optimization on all 3D Gaussian points in the local region, update the center position, covariance matrix, opacity and spherical harmonic coefficients of the 3D Gaussian points, and perform pruning operations on 3D Gaussian points that are distributed outside the local region and are judged to be redundant to control the model size. S3.3.4 Synchronization of Composite Index: After a partial update is completed, the list of Gaussian points in the corresponding region of the composite index is updated according to the actual spatial range occupied by the optimized Gaussian points to ensure index consistency; S3.4 Dynamic Embedding of Shooting Path: After the local 3DGS model is updated, the precise position and orientation of the target marker are identified based on the updated model. The specific identification methods include: rendering the image of the marker area through the 3DGS model, performing template matching or feature point comparison to determine the precise spatial pose of the marker, and using the precise pose of the marker as a sub-target point for local path planning, embedding it into the intermediate results of path planning.

[0013] As a further preferred embodiment of the present invention, step S4 specifically includes: After the marker pose recognition is completed and the local 3DGS model is updated, local fine path planning is performed to generate the optimal path from the current drone position to the optimal shooting position of the marker. S4.1 Generation of Candidate Shooting Locations: Based on the size, orientation, and camera intrinsic parameters of the marker, generate a set of candidate shooting locations around the marker. Each candidate shooting location Includes position coordinates And the desired attitude of the drone; S4.2 Local Cost Map Construction: In local areas Internally, the 3D occupancy voxels of the point cloud model are fused with the semantic information provided by the 3D Gaussian splash model to construct a local cost map, in which the cost of each voxel v is... Defined as: in, Occupancy cost represents the risk of a drone colliding with voxel v while flying over it. Application cost: reflects the correlation between voxel v and the shooting task; Distance Cost: Voxel v and Candidate Shooting Points The Euclidean distance is used to guide the target when the path search ends at a specific candidate point; , , These are the adjustable coefficients for each weight. ; Specifically, the occupancy probability of voxel v in the point cloud model PointMap is... The value is determined to be high, indicating a greater risk of collision, and falls between 0 and 1. To avoid path oscillations caused by small fluctuations in the probability value around 0.5, this invention employs a piecewise nonlinear mapping function: Among them, by This is the idle threshold, considered to be within a safe range. This is the occupancy threshold; anything above this value is considered a rigid obstacle, and the voxel is directly blocked during path search. Set to block this voxel, with the maximum penalty value. ; ; This is a scaling factor used to adjust the steepness of the cost in the transition region; for unobserved voxels or regions, it is set... It neither completely blocks paths nor imposes moderate penalties to drive paths to prioritize safe areas already mapped.

[0014] S4.3 Multi-candidate point path search and optimization: Taking the current position of the drone as the starting point, each candidate shooting position is searched and optimized. As the target point, a path search is performed on the local cost map; a comprehensive cost evaluation is conducted for each path, and the formula for calculating the comprehensive cost is as follows: in, The path length represents the distance from the drone's current location to the candidate shooting location. The total spatial length of the planned path is calculated as the sum of the Euclidean distances between all adjacent nodes on the path. For path smoothness, the supplementary angle scalar is calculated by taking the cosine of the angle between adjacent direction vectors of the UAV. The image coverage quality of markers along the path is obtained by cumulatively evaluating the visibility of markers rendered along the path by the 3DGS model, i.e., along the candidate path. Uniform sampling A discrete point is used to generate a virtual viewpoint rendering image at the sampling point. The ratio of the total area of ​​pixels projected by the 3D Gaussian point corresponding to the marker in the rendering image to the total area of ​​the image is calculated. , , These are the adjustable coefficients for each weight. Select the candidate shooting position v and its corresponding path that minimizes the overall cost. As the optimal shooting path; S4.4 Path Execution and Feedback Control: Controlling the UAV along the optimal path The system flies to the selected shooting location and performs the task of acquiring marker images. During the flight, it performs real-time rendering based on the local 3DGS model to generate a predicted image from the current viewpoint. The difference between the predicted image and the real-time image is used to determine whether there are dynamic obstacles that have not been mapped. If the difference exceeds the set threshold, the S3 process is triggered to be executed again, realizing a closed-loop real-time scene update and adaptive replanning.

[0015] A system employing the aforementioned hierarchical path planning method for indoor UAVs based on a joint model of 3D Gaussian splashing and point clouds includes the following modules: Data acquisition and mapping module: mounted on a drone, used to acquire laser point cloud data and RGB-D image sequences, and construct a dual-mode joint map. The dual-mode joint map includes a point cloud model and a three-dimensional Gaussian splash model, and also includes a joint index that establishes the spatial mapping relationship between the two. Layered path planning module: connected to the data acquisition and mapping module, used to divide the flight airspace into multiple altitude layers based on the altitude priority principle, and generate a high-altitude priority global path from the starting position to the target marker; this module further includes an altitude layer management submodule, a main channel path planning submodule, and an end approach path submodule; Trigger Judgment and Positioning Module: Connected to the hierarchical path planning module, it is used to monitor the distance between the UAV and the target marker in real time. When the distance is less than a preset threshold, a local update mechanism is triggered to determine the precise pose of the marker. Incremental mapping and updating module: connected to the trigger judgment and positioning module, used to incrementally update the three-dimensional Gaussian splash model in a local area near the target marker. The incremental update includes incremental addition of Gaussian points, parameter optimization and pruning. Local path planning and optimization module: connected to the incremental mapping and updating module, used to generate multiple candidate shooting locations based on the updated 3D Gaussian splash model, construct a local cost map and select the optimal path; Flight control and execution module: connected to the local path planning and optimization module, used to control the UAV to fly along the optimal path to the shooting position to perform the task, and to perform dynamic obstacle perception and replanning triggering based on real-time rendering of the 3DGS model during flight; Model maintenance module: Connected to the data acquisition and mapping module and the incremental mapping and updating module, it is used for long-term incremental maintenance, index synchronization and anomaly detection of point cloud model and 3D Gaussian splash model.

[0016] Compared with the prior art, the beneficial effects of the present invention are: (1) Dual-mode complementary advantages, balancing geometric accuracy and visual realism: For the first time, a laser point cloud model and a 3D Gaussian splash vision model are jointly applied to indoor UAV path planning within the same framework. The point cloud model provides high-precision geometric collision information for safe distance constraints, while the 3DGS model provides high-fidelity texture information for marker recognition and tracking, thus solving the bottleneck of the applicability of single representation methods in complex indoor scenes. In existing methods, either only point clouds are used (lacking visual information to support task execution) or only 3DGS is used (geometric collision information is unclear). This invention achieves complementary advantages through dual-mode combination.

[0017] (2) "High-altitude priority and hierarchical planning" improves the efficiency of global path planning: Unlike existing technologies (such as Splat-Nav, GRAD-NAV, etc.) that use equal weighting for planar planning of ground robots in the entire space, this invention proposes a height-hierarchical planning strategy based on the vertical characteristics of indoor airspace. The high-altitude layer is used as the global main channel for rapid coverage flight, while the low-altitude layer is used for fine shooting operations. In typical indoor scenarios (such as buildings with a floor height of 3 m), high-altitude priority planning can effectively avoid obstacle avoidance overhead in densely populated low-altitude areas.

[0018] (3) The local 3D Gaussian splash incremental update mechanism achieves a balance between real-time response and high-fidelity reconstruction: Existing technologies usually either rebuild the 3DGS model only once before the mission (which cannot adapt to local environmental changes) or use global real-time continuous reconstruction (which involves excessive computation and affects flight performance). This invention innovatively proposes a mechanism that triggers local 3DGS incremental updates only near markers, which ensures real-time response while taking into account the effective use of computing resources, providing a feasible technical path for high-fidelity navigation in large-scale indoor missions.

[0019] (4) High versatility and compatibility with multiple sensor configurations and robot platforms: This invention does not rely on specific sensor or computing platform configurations. It can be deployed on high-end UAVs equipped with LiDAR and RGB-D cameras, and can also be adapted to lightweight platforms with only visual sensors through depth estimation algorithms. It has good system compatibility and scalability. Attached Figure Description

[0020] Figure 1 This is a schematic diagram illustrating the implementation process of the indoor UAV hierarchical path planning method based on a joint model of three-dimensional Gaussian splash and point cloud provided by the present invention. Detailed Implementation

[0021] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0022] To achieve the above objectives, this invention provides a hierarchical path planning method for indoor unmanned aerial vehicles (UAVs) based on a joint model of 3D Gaussian splashing and point cloud. This patent specifically includes the following steps: S1: Dual-mode joint map pre-construction S1.1 Data Acquisition and Preprocessing: The drone is controlled to traverse the target indoor area along a preset scanning path, collecting point cloud data of the environment using the onboard LiDAR, and simultaneously acquiring RGB images and depth images from various viewpoints using an RGB-D camera. The point cloud data is denoted as... Each point contains three-dimensional coordinates. and possible reflection intensity information; the RGB image sequence is denoted as The corresponding depth image is denoted as The RGB-D camera simultaneously records the camera pose during acquisition.

[0023] S1.2 Constructing the Point Cloud Model: The raw point cloud acquired by the LiDAR is filtered (e.g., voxel filtering to remove outliers) to obtain a dense point cloud model. On this point cloud model, an occupancy grid map is constructed using voxelization, where 0 represents free space and 1 represents complete obstacle occupation. The point cloud model provides high-precision geometric collision detection information for the UAV, used for safe distance verification in global path planning.

[0024] S1.3 Constructing the initial 3D Gaussian splash model: Using multi-view image sequences and corresponding depth images acquired by an RGB-D camera, a 3DGS initial model of the indoor scene is constructed through a 3D Gaussian splash reconstruction process. After optimization operations such as densification and pruning, a high-fidelity visual model of the indoor scene is obtained, which can provide rich texture and color information for drones to support the recognition and tracking of markers.

[0025] S1.4 Establish a dual-model joint map index: Establish a spatial mapping index between the point cloud model and the 3D Gaussian splash model to ensure that in the world coordinate system... any spatial location The corresponding voxel occupancy information and Gaussian point information can be quickly found here.

[0026] S2: Global path planning based on a high-altitude priority hierarchical strategy The core innovation of this invention lies in the introduction of a global path planning strategy based on flight altitude hierarchy. Unlike existing technologies, this method divides the indoor space into high-altitude priority channels and low-altitude fine-grained areas, and dynamically determines the planning strategy based on mission objectives and flight altitude.

[0027] S2.1 Establish a spatial height-level model: Based on the actual structure of the indoor environment (such as room height, obstacle distribution, etc.) and the mission requirements of the UAV, the flight airspace is divided into L height levels along the vertical direction, denoted as LL. ,in The top floor (near the ceiling). The lowest layer (closest to the ground) is defined by an adaptive division of height layers based on the point cloud distribution density of the indoor environment. Areas with dense obstacles have denser height layers, while open areas have sparser height layers.

[0028] S2.2 Setting the High-Altitude Priority Planning Principle: This method sets the high-altitude layer as the main channel layer for global path planning, denoted as... (usually the highest floor) The most open layer (or the least obstructed layer) has the following characteristics: fewest obstacles, best visibility, and fastest flight speed. In global path planning, the drone is preferentially guided to this layer. To conduct long-distance coverage flights.

[0029] S2.3 Generate a high-altitude priority global path: Receive task instructions from the user, which include a set of one or more target marker locations. And marker attributes (such as marker type, color, texture features, etc.), for each target marker Perform the following steps: S2.3.1 Altitude Conversion Between Start and End Points: Let the current position of the UAV in the world coordinate system {W} be... The target waypoint is .like Located in the low-altitude region and If the distance is too far, a layer from the current altitude to the main high-altitude passage will be generated. Height adjustment path This path prioritizes vertical ascent to avoid long horizontal crossings.

[0030] S2.3.2 Main Channel Layer Path Planning: In the context of... On a plane centered on the height, using the occupancy information planned by the point cloud model, A The algorithm generates an unobstructed global path from the starting projection point to the target projection point. During path search, the cost function for expanding nodes is defined as: Where g(n) is the actual flight cost from the starting point to node n (combining distance, time, and other indicators); h(n) is the heuristically estimated cost from node n to the target point (often using Euclidean distance); and sep(n) is the safe separation cost, which is based on the distance between node n and the nearest voxel obstacle. Calculation, defined as The aim is to guide the path as far away from obstacles as possible; These are adjustable weighting coefficients.

[0031] S2.3.3 Terminal Altitude Descent Path Planning: This involves the UAV flying towards the target marker. Nearby (typically defined as a horizontal projection distance from the target location less than a threshold) For example, at 3m, generate from the high-altitude main channel layer. Sink to the height of the target marker descent path The descent path is preferentially executed along the vertical direction, and rapid collision detection is performed on the voxels along the path.

[0032] S2.3.4 Global Path Composition: [This part is incomplete and requires more context to translate accurately.] and The three paths are pieced together in sequence to form... arrive Full global path For multiple target markers, the Traveling Salesman Problem optimization algorithm is used to determine the optimal access order, and then the global path sequence is synthesized sequentially.

[0033] S3: Real-time local scene updates based on 3D Gaussian splashing When the drone flies to the vicinity of the target marker (distance from the target marker) The horizontal projection distance is less than , When the value is typically between 2 m and 5 m, a local scene real-time update mechanism is activated to incrementally update the 3DGS model for the local area around the target marker.

[0034] S3.1 Detection trigger condition: The system monitors the drone and the current target marker in real time. Horizontal projection distance between .when At that time, a local scene update process is triggered. These are parameters that are adaptively set based on the camera's maximum effective sensing distance and can be dynamically adjusted through online calibration.

[0035] S3.2 Local area demarcation: using target markers Set the safe distance radius for drones at the center. The local safety buffer radius of the drone And delineate local update areas. The following conditions must be met: S3.3 Incremental 3D Gaussian Splash Mapping: Unlike the overall reconstruction in S1.3, this step only reconstructs local areas. Perform remodeling or optimization updates, rather than a global rebuild. Specific sub-steps include: S3.3.1 Data Acquisition and Pose Optimization: The UAV hovers or performs minor maneuvers around the target marker, continuously acquiring RGB-D image sequences. Furthermore, under the guidance of the joint map index, the camera pose estimation is optimized in real time. The 3DGS model is used to render the 3D Gaussian point set in the current area, and the rendered image is matched with the real image to optimize the pose. This is a key technical feature for improving the speed and accuracy of local mapping.

[0036] S3.3.2 Incremental addition of 3D Gaussian points: Identify local regions based on newly acquired image data. Regions not yet covered by the existing 3DGS model or with insufficient reconstruction quality are identified. For these regions, 3D Gaussian point clouds are initialized and interpolated into the 3DGS model. The parameters of the interpolated 3D Gaussian points are optimized by minimizing the loss function between the rendered image and the ground truth image.

[0037] S3.3.3 Parameter Optimization and Pruning of 3D Gaussian Points: Several rounds of iterative optimization are performed on all Gaussian points within the local region, updating the center position, covariance matrix, opacity, and spherical harmonic coefficients of the 3D Gaussian points. For 3D Gaussian points distributed outside the local region and deemed redundant, pruning operations are performed to control the model size.

[0038] S3.3.4 Synchronization of Composite Index: After the local update is completed, the list of 3D Gaussian points in the corresponding region of the composite index is updated according to the actual spatial range occupied by the optimized 3D Gaussian points to ensure the consistency of the index.

[0039] S3.4 Dynamic Embedding of the Shooting Path: After updating the local 3D Gaussian splash model, the precise position and orientation of the target marker are identified based on the updated model. Specific identification methods include, but are not limited to: rendering an image of the marker area using the 3DGS model, performing template matching or feature point comparison to determine the precise spatial pose of the marker. The precise pose of the marker is then used as a sub-target point in the local path planning and embedded into the intermediate path planning results.

[0040] S4: Local Fine-grained Path Planning After the marker pose recognition is completed and the local 3DGS model is updated, local fine path planning is performed to generate the optimal path from the current UAV position to the optimal shooting position of the marker.

[0041] S4.1 Generation of Candidate Shooting Locations: Based on the size, orientation, and camera intrinsic parameters of the marker, generate a set of candidate shooting locations around the marker. Each candidate shooting location Includes position coordinates And the desired attitude of the drone; S4.2 Local Cost Map Construction: In local areas Internally, the 3D occupancy voxels of the point cloud model are fused with the semantic information provided by the 3DGS model to construct a local cost map, in which the cost of each voxel v is calculated. Defined as: in, Occupancy cost represents the risk of a drone colliding with voxel v while flying over it. Application cost: reflects the correlation between voxel v and the shooting task; Distance Cost: Voxel v and Candidate Shooting Points The Euclidean distance is used to guide the target when the path search ends at a specific candidate point; , , These are the adjustable coefficients for each weight. ; Specifically, the occupancy probability of voxel v in the point cloud model PointMap is... The value is determined to be high, indicating a greater risk of collision, and falls between 0 and 1. To avoid path oscillations caused by small fluctuations in the probability value around 0.5, this invention employs a piecewise nonlinear mapping function: Among them, by This is the idle threshold, considered to be within a safe range. This is the occupancy threshold; anything above this value is considered a rigid obstacle, and the voxel is directly blocked during path search. Set to block this voxel, with the maximum penalty value. ; ; This is a scaling factor used to adjust the steepness of the cost in the transition region; for unobserved voxels or regions, it is set... It neither completely blocks paths nor imposes moderate penalties to drive paths to prioritize safe areas already mapped.

[0042] S4.3 Multi-candidate point path search and optimization: Taking the current position of the drone as the starting point, each candidate shooting position is searched and optimized. As the target point, a path search is performed on the local cost map; a comprehensive cost evaluation is conducted for each path, and the formula for calculating the comprehensive cost is as follows: in, The path length represents the distance from the drone's current location to the candidate shooting location. The total spatial length of the planned path is calculated as the sum of the Euclidean distances between all adjacent nodes on the path. For path smoothness, the supplementary angle scalar is calculated by taking the cosine of the angle between adjacent direction vectors of the UAV. The image coverage quality of markers along the path is obtained by cumulatively evaluating the visibility of markers rendered along the path by the 3DGS model, i.e., along the candidate path. Uniform sampling A discrete point is used to generate a virtual viewpoint rendering image at the sampling point. The ratio of the total area of ​​pixels projected by the 3D Gaussian point corresponding to the marker in the rendering image to the total area of ​​the image is calculated. , , These are the adjustable coefficients for each weight. Select the candidate shooting position v and its corresponding path that minimizes the overall cost. As the optimal shooting path.

[0043] S4.4 Path Execution and Feedback Control: Controlling the UAV along the optimal path The system flies to the selected shooting location and performs the task of acquiring marker images. During flight, it performs real-time rendering based on a local 3D Gaussian splash model to generate a predicted image from the current viewpoint. The difference between the predicted image and the real-time image is used to determine whether there are any unmapped dynamic obstacles. If the difference exceeds a set threshold, the S3 process is triggered to be executed again, realizing a closed-loop real-time scene update and adaptive replanning.

[0044] To achieve the above method, the present invention further provides an indoor UAV hierarchical path planning system based on a joint model of visual 3D Gaussian splash and point cloud, comprising the following modules: (1) Data acquisition and mapping module: mounted on a drone, used to acquire laser point cloud data and RGB-D image sequences, and to construct a dual-mode joint map. The dual-mode joint map includes a point cloud model and a three-dimensional Gaussian splash model, and also includes a joint index that establishes the spatial mapping relationship between the two.

[0045] (2) Layered path planning module: connected to the data acquisition and mapping module, used to divide the flight airspace into multiple altitude layers based on the altitude priority principle, and generate a high-altitude priority global path from the starting position to the target marker; this module further includes an altitude layer management submodule, a main channel path planning submodule and an end approach path submodule.

[0046] (3) Trigger Judgment and Positioning Module: Connected to the hierarchical path planning module, it is used to monitor the distance between the UAV and the target marker in real time. When the distance is less than a preset threshold, a local update mechanism is triggered, and the precise pose of the marker is determined.

[0047] (4) Incremental mapping and updating module: connected to the trigger judgment and positioning module, used to incrementally update the three-dimensional Gaussian splash model in a local area near the target marker. The incremental update includes incremental addition of Gaussian points, parameter optimization and pruning operations.

[0048] (5) Local path planning and optimization module: connected to the incremental mapping and updating module, used to generate multiple candidate shooting locations based on the updated three-dimensional Gaussian splash model, construct a local cost map and select the optimal path.

[0049] (6) Flight control and execution module: connected to the local path planning and optimization module, used to control the UAV to fly along the optimal path to the shooting position to perform the task, and to perform dynamic obstacle perception and replanning triggering based on real-time rendering of the 3DGS model during flight.

[0050] (7) Model maintenance module: connected to the data acquisition and mapping module and the incremental mapping and updating module, used for long-term incremental maintenance, index synchronization and anomaly detection of point cloud model and three-dimensional Gaussian splash model.

[0051] It will be apparent to those skilled in the art that the present invention is not limited to the details of the exemplary embodiments described above, and that the invention can be implemented in other specific forms without departing from its spirit or essential characteristics. Therefore, the embodiments should be considered in all respects as exemplary and non-limiting, and the scope of the invention is defined by the appended claims rather than the foregoing description. Thus, all variations falling within the meaning and scope of equivalents of the claims are intended to be included within the present invention. No reference numerals in the claims should be construed as limiting the scope of the claims.

[0052] Furthermore, it should be understood that although this specification describes embodiments, not every embodiment contains only one independent technical solution. This narrative style is merely for clarity. Those skilled in the art should consider the specification as a whole, and the technical solutions in each embodiment can also be appropriately combined to form other embodiments that can be understood by those skilled in the art.

Claims

1. A method for layered path planning of indoor unmanned aerial vehicle based on three-dimensional Gaussian splash and point cloud joint model, characterized in that, Includes the following steps: S1: Dual-mode joint map pre-construction: Control the UAV to traverse the target area, use LiDAR to collect point cloud data, and use an RGB-D camera to collect RGB and depth images; filter the point cloud data to construct a point cloud model, and use voxelization to construct an occupied grid map; use multi-view images and depth images collected by the RGB-D camera to construct an initial 3D Gaussian splash model of the indoor scene through a 3D Gaussian splash reconstruction process; establish a spatial mapping index between the point cloud model and the 3D Gaussian splash model; S2: Global path planning based on high-altitude priority layering strategy: Divide the flight airspace vertically into L altitude layers; set the high-altitude layer as the main channel layer for global path planning; receive task instructions containing the location and attributes of target markers, and generate a high-altitude priority global path from the starting point to the target markers; S3: Real-time local scene update based on 3D Gaussian splash: When the horizontal projection distance between the UAV and the target marker is less than or equal to a preset threshold, a local update area is defined with the target marker as the center, and the 3D Gaussian splash model is incrementally updated in this area. S4: Local fine path planning: After the marker pose recognition is completed and the local 3D Gaussian splash model is updated, the optimal path from the current position of the UAV to the optimal shooting position of the marker is generated and executed. 2.The indoor UAV hierarchical path planning method based on three-dimensional Gaussian splash and point cloud combined model according to claim 1, wherein, Step S1 specifically includes: S1.1 Data collection and pre-processing: control the UAV to traverse the target indoor area along the preset scanning path, use the laser radar carried to collect the point cloud data of the environment, and use the RGB-D camera to collect the RGB image and depth image of each view angle, the point cloud data is denoted as , where each point contains three-dimensional coordinates (x, y, z) and possible reflection intensity information; the RGB image sequence is denoted as , and the corresponding depth image is denoted as , and the camera pose is recorded synchronously when the RGB-D camera collects. S1.2 Constructing a point cloud model: The original point cloud collected by the lidar is filtered to obtain a dense point cloud model. On this point cloud model, a three-dimensional grid map is constructed using the voxelization method, where 0 represents free space and 1 represents complete occupation by obstacles. The point cloud model provides high-precision geometric collision detection information for the UAV and is used for safe distance verification in global path planning. S1.3 Constructing the initial 3D Gaussian Splatting model: Using multi-view image sequences and corresponding depth images acquired by an RGB-D camera, a 3D Gaussian Splatting (3DGS) reconstruction process is used to construct an initial 3DGS model of the indoor scene. After densification and pruning optimization operations, a high-fidelity visual model of the indoor scene is obtained, providing texture and color information for the drone to support the identification and tracking of markers. S1.4 Establish a dual-model joint map index: Establish a spatial mapping index between the point cloud model PointMap and the 3D Gaussian splash model 3DGSMap to ensure that in the world coordinate system... any spatial location The corresponding voxel occupancy information and Gaussian point information can be quickly found.

3. The indoor UAV hierarchical path planning method based on a joint model of 3D Gaussian splash and point cloud as described in claim 2, characterized in that, Step S2 specifically includes: S2.1 Establishing a spatial height hierarchy model: Based on the actual structure of the indoor environment and the mission requirements of the UAV, the flight airspace is divided into L height layers along the vertical direction, denoted as LL. ,in The highest level The lowest layer is defined by adaptively dividing the height layers based on the point cloud distribution density of the indoor environment. Areas with dense obstacles have denser height layers, while open areas have sparser height layers. S2.2 Setting the High-Altitude Priority Planning Principle: The high-altitude layer is designated as the main channel layer for global path planning, denoted as... In global path planning, priority is given to guiding the drone to... Conduct long-range coverage flights; S2.3 Generates a high-altitude priority global path: Receives task instructions from the user, which include a set of one or more target marker locations. And the attributes of the marker, for each target marker Perform the following steps: S2.3.1 Altitude Conversion Between Start and End Points: Assume the UAV is currently in the world coordinate system. The position in the middle is The target waypoint is ,like Located in the low-altitude region and If the distance is too far, a layer from the current altitude to the main high-altitude passage will be generated. Height adjustment path This path prioritizes vertical ascent to avoid long horizontal crossings; S2.3.2 Main Channel Layer Path Planning: In the context of... On a plane centered on the height, using the occupancy information planned by the point cloud model, A The algorithm generates an unobstructed global path from the starting projection point to the target projection point. During path search, the cost function for expanding nodes is defined as: in, This represents the actual flight cost from the starting point to node n. For heuristic cost estimation from node n to the target point; The cost for safe separation is based on the distance between node n and the nearest voxel obstacle. Calculation, defined as The aim is to guide the path as far away from obstacles as possible; These are adjustable weighting coefficients; S2.3.3 Terminal Altitude Descent Path Planning: This involves the UAV flying towards the target marker. When nearby, The horizontal projection distance from the target location is less than a threshold. Generate from the high-altitude main channel layer Sink to the height of the target marker descent path The descent path is executed primarily along the vertical direction, and rapid collision detection is performed on the voxels along the path. S2.3.4 Global Path Composition: and The three paths are pieced together in sequence to form... arrive Full global path For multiple target markers, the Traveling Salesman Problem optimization algorithm is used to determine the optimal access order, and then the global path sequence is synthesized sequentially. S3: Real-time local scene updates based on 3D Gaussian splashing When the drone flies to the vicinity of the target marker, that is, at close range to the target marker The horizontal projection distance is less than A local scene real-time update mechanism is initiated to incrementally update the 3DGS model for the local area around the target marker. S3.1 Detection trigger condition: The system monitors the drone in real time. With the current target marker Horizontal projection distance between ， when At that time, a partial scene update process is triggered. These parameters are adaptively set based on the camera's maximum effective sensing distance and are dynamically adjusted through online calibration. S3.2 Local area demarcation: using target markers Set the safe distance radius for drones at the center. The local safety buffer radius of the drone And delineate local update areas. The following conditions must be met: S3.3 Incremental 3D Gaussian splash mapping: for local areas The specific sub-steps for remodeling or optimization updates include: S3.3.1 Data Acquisition and Pose Optimization: The UAV hovers or performs minor maneuvers around the target marker, continuously acquiring RGB-D image sequences. And under the guidance of the joint map index, the camera pose estimation is optimized in real time. The 3DGS model is used to render the 3D Gaussian point set in the current area, and the rendered image is matched with the real image to optimize the pose. S3.3.2 Incremental addition of 3D Gaussian points: Identify local regions based on newly acquired image data. For regions that are not yet covered by the existing Gaussian model or whose reconstruction quality is insufficient, 3D Gaussian point clouds are initialized and inserted into the 3DGS model. The parameters of the inserted 3D Gaussian points are optimized by minimizing the loss function between the rendered image and the real image. S3.3.3 Parameter optimization and pruning of 3D Gaussian points: Perform several rounds of iterative optimization on all 3D Gaussian points in the local region, update the center position, covariance matrix, opacity and spherical harmonic coefficients of the 3D Gaussian points, and perform pruning operations on 3D Gaussian points that are distributed outside the local region and are judged to be redundant to control the model size. S3.3.4 Synchronization of Composite Index: After the local update is completed, the 3D Gaussian point list of the corresponding region of the composite index is updated according to the actual spatial range occupied by the optimized Gaussian points to ensure the consistency of the index; S3.4 Dynamic Embedding of Shooting Path: After the local 3DGS model is updated, the precise position and orientation of the target marker are identified based on the updated model. The specific identification methods include: rendering the image of the marker area through the 3DGS model, performing template matching or feature point comparison to determine the precise spatial pose of the marker, and using the precise pose of the marker as a sub-target point for local path planning, embedding it into the intermediate results of path planning.

4. The indoor UAV hierarchical path planning method based on a joint model of 3D Gaussian splash and point cloud as described in claim 2, characterized in that, Step S4 specifically includes: After the marker pose recognition is completed and the local 3DGS model is updated, local fine path planning is performed to generate the optimal path from the current drone position to the optimal shooting position of the marker. S4.1 Candidate Shooting Location Generation: Based on the size, orientation, and camera intrinsic parameters of the marker, generate a set of candidate shooting locations around the marker. Each candidate shooting location Includes position coordinates And the desired attitude of the drone; S4.2 Local Cost Map Construction: In local areas Internally, the 3D occupancy voxels of the point cloud model are fused with the semantic information provided by the 3D Gaussian splash model to construct a local cost map, in which the cost of each voxel v is... Defined as: in, Occupancy cost represents the level of risk of a collision when the drone flies over voxel v. Application cost: reflects the correlation between voxel v and the shooting task; Distance Cost: Voxel v and Candidate Shooting Points The Euclidean distance is used to guide the target when the path search ends at a specific candidate point; , , These are the adjustable coefficients for each weight. ; Specifically, the occupancy probability of voxel v in the point cloud model PointMap is... The value is determined to be high, indicating a greater risk of collision, and falls between 0 and 1. To avoid path oscillations caused by small fluctuations in the probability value around 0.5, this invention employs a piecewise nonlinear mapping function: Among them, by This is the idle threshold, considered to be within a safe range. This is an occupancy threshold; anything exceeding this value is considered a rigid obstacle, and the voxel is directly blocked during path search. Set to block this voxel, with the maximum penalty value. ; ; This is a scaling factor used to adjust the steepness of the cost in the transition region; for unobserved voxels or regions, it is set... It neither completely blocks paths nor imposes moderate penalties to drive paths to prioritize safe areas in the established map. S4.3 Multi-candidate point path search and optimization: Taking the current position of the drone as the starting point, each candidate shooting position is searched and optimized. As the target point, a path search is performed on the local cost map; a comprehensive cost evaluation is conducted for each path, and the formula for calculating the comprehensive cost is as follows: in, The path length represents the distance from the drone's current location to the candidate shooting location. The total spatial length of the planned path is calculated as the sum of the Euclidean distances between all adjacent nodes on the path. For path smoothness, the supplementary angle scalar is calculated by taking the cosine of the angle between adjacent direction vectors of the UAV. The image coverage quality of markers along the path is obtained by cumulatively evaluating the visibility of markers rendered along the path by the 3DGS model, i.e., along the candidate path. Uniform sampling A discrete point is used to generate a virtual viewpoint rendering image at the sampling point. The ratio of the total area of ​​pixels projected by the Gaussian point corresponding to the marker in the rendering image to the total area of ​​the image is calculated. , , These are the adjustable coefficients for each weight. ; Select the candidate shooting position v and its corresponding path that minimizes the overall cost. As the optimal shooting path; S4.4 Path Execution and Feedback Control: Controlling the UAV along the optimal path The system flies to the selected shooting location and performs the task of acquiring marker images. During the flight, it performs real-time rendering based on a local 3D Gaussian splash model to generate a predicted image from the current viewpoint. The difference between the predicted image and the real-time image is used to determine whether there are any unmapped dynamic obstacles. If the difference exceeds a set threshold, the S3 process is triggered to be executed again, thus achieving closed-loop real-time scene update and adaptive replanning.

5. A system employing the indoor UAV hierarchical path planning method based on a joint model of three-dimensional Gaussian splashing and point cloud as described in any one of claims 1-4, characterized in that, Includes the following modules: Data acquisition and mapping module: mounted on a drone, used to acquire laser point cloud data and RGB-D image sequences, and construct a dual-mode joint map. The dual-mode joint map includes a point cloud model and a three-dimensional Gaussian splash model, and also includes a joint index that establishes the spatial mapping relationship between the two. Layered path planning module: connected to the data acquisition and mapping module, used to divide the flight airspace into multiple altitude layers based on the altitude priority principle, and generate a high-altitude priority global path from the starting position to the target marker; this module further includes an altitude layer management submodule, a main channel path planning submodule, and an end approach path submodule; Trigger Judgment and Positioning Module: Connected to the hierarchical path planning module, it is used to monitor the distance between the UAV and the target marker in real time. When the distance is less than a preset threshold, a local update mechanism is triggered to determine the precise pose of the marker. Incremental mapping and updating module: connected to the trigger judgment and positioning module, used to incrementally update the three-dimensional Gaussian splash model in a local area near the target marker. The incremental update includes incremental addition of Gaussian points, parameter optimization and pruning. Local path planning and optimization module: connected to the incremental mapping and updating module, used to generate multiple candidate shooting locations based on the updated 3D Gaussian splash model, construct a local cost map and select the optimal path; Flight control and execution module: connected to the local path planning and optimization module, used to control the UAV to fly along the optimal path to the shooting position to perform the task, and to perform dynamic obstacle perception and replanning triggering based on real-time rendering of the 3DGS model during flight; Model maintenance module: Connected to the data acquisition and mapping module and the incremental mapping and updating module, it is used for long-term incremental maintenance, index synchronization and anomaly detection of point cloud model and 3D Gaussian splash model.

Images