A building unmanned aerial vehicle inspection route intelligent planning method based on reinforcement learning

Abstract

The application discloses a kind of based on reinforcement learning's building unmanned plane inspection route intelligent planning method, comprising the following steps: collecting building facade contour information data, generates three-dimensional semantic model;Based on three-dimensional semantic model, simulation environment is constructed, and dynamic inspection virtual scene is formed;Hierarchical reinforcement learning model is established in dynamic inspection virtual scene, and multiple candidate path set is formed;Introduce voxel hash ray space index, output route ruling document;Based on route ruling document, form continuous controllable executable track set;Executable track set is imported into unmanned plane control system, and generates task preparation file;According to task preparation file, route flight inspection task is executed, and task execution log is output;Task execution log is uploaded to cloud, track deviation is analyzed and counted, and intelligent planning and adaptive closed-loop evolution are realized.The application realizes building unmanned plane inspection route intelligent planning.

Description

Technical Field

[0001] This invention relates to the field of unmanned aerial vehicle (UAV) inspection technology, and in particular to an intelligent planning method for UAV inspection routes in buildings based on reinforcement learning. Background Technology

[0002] With the continuous improvement of intelligent construction and operation and maintenance, drones have been widely used in areas such as facade inspection and curtain wall inspection of high-rise buildings. Traditional drone inspection route planning methods mostly rely on manually setting waypoints or path calculation based on static geometric models, and their trajectory generation is often based on fixed rules or heuristic algorithms. These algorithms can obtain relatively stable inspection paths in static environments, but they are difficult to cope with the influence of multiple factors such as complex building structures, dynamic changes in obstacle distribution, and wind interference.

[0003] Existing technologies have also explored applying machine learning to UAV path optimization, but most methods remain at the stage of single-layer reinforcement learning or local path correction based on reward functions, making it difficult to achieve adaptive control of global flight path planning. Especially in building inspection scenarios, where building surfaces are complex and spaces are narrow, single-layer reinforcement learning models have low convergence efficiency under multi-dimensional input conditions, resulting in high algorithm training costs and insufficient generalization ability.

[0004] Meanwhile, existing trajectory planning systems are typically isolated from flight control systems, lacking a data feedback mechanism after mission execution and failing to achieve a self-closing learning process from execution deviations to strategy optimization. Due to the lack of an effective path for mission log analysis and strategy retraining, the system struggles to self-correct voxel index parameters or reinforcement learning model decision-making strategies using historical data. This results in trajectory planning remaining in its initial state after multiple mission iterations, unable to continuously optimize in response to changes in environment and equipment performance.

[0005] Therefore, how to provide a method for intelligent planning of unmanned aerial vehicle (UAV) inspection routes for buildings based on reinforcement learning is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0006] One objective of this invention is to propose an intelligent planning method for building inspection routes using unmanned aerial vehicles (UAVs) based on reinforcement learning. By introducing a hierarchical reinforcement learning model and voxel hash ray spatial indexing, this invention achieves dynamic intelligent planning of inspection routes for building UAVs. It can automatically generate safe, efficient, and continuously controllable inspection tracks in complex building environments and perform parameter self-calibration and strategy retraining based on task execution logs. This significantly improves the accuracy of UAV route planning, energy utilization, and environmental adaptability, while reducing the risks of human intervention and task execution.

[0007] A method for intelligent planning of unmanned aerial vehicle (UAV) inspection routes for buildings based on reinforcement learning, according to an embodiment of the present invention, includes the following steps: Collect building facade outline information data, perform semantic segmentation on the building facade outline, generate a three-dimensional semantic model, and extract visible field constraints and risk area information at the same time; A simulation environment is built based on a three-dimensional semantic model to form an interactive, dynamic inspection virtual scene for subsequent strategy training and verification. In a dynamic inspection virtual scenario, a hierarchical reinforcement learning model is established. The upper-level strategy generates a building area-level flight segment sequence and time arrangement based on the semantic features of the inspection target point, while the lower-level strategy combines the environmental state to generate local micro-tracks and attitude control commands, forming a set of multiple candidate paths. A voxel hash ray spatial index is introduced into the multi-candidate path set. Ray channel decision is performed on each candidate track. A channel registration table containing channel number, attitude window and timestamp is constructed and the route decision file is output. Based on the route decision documents, the channel tracks are masked, and only the executable tracks confirmed by the index are retained, forming a continuous and controllable set of executable tracks; Import the set of executable flight paths into the UAV control system, initialize flight mission parameters, verify the integrity of the flight path and energy requirements, and generate mission preparation files. Perform flight inspection tasks according to the mission preparation documents, dynamically adjust the flight path and output the mission execution log; The task execution logs are uploaded to the cloud, and the trajectory deviations are analyzed and statistically analyzed. The results are used to calibrate the voxel hash ray spatial index parameters and retrain the strategy, so as to realize the intelligent planning system and adaptive closed-loop evolution.

[0008] Optionally, the generation process of the three-dimensional semantic model specifically includes: Import building information files, read the geometric data and attribute information of building components, establish a global coordinate system, set a unified origin, direction and unit, transform the local coordinates of the building to global coordinates, extract the building facade outline data to obtain inspection target points, and resample the point cloud according to the set density. Determine the work space range according to the outer boundary, and divide it into a uniform cubic voxel grid according to the set voxel side length. Record the voxel number and spatial position parameters of each sampling point, and generate an initial dataset containing point cloud coordinates, voxel number and spatial boundary. Establish a semantic category set required for building inspection tasks, perform semantic segmentation on the point cloud of the initial dataset point by point, assign a unique category label to each point, aggregate the segmentation results by voxel number, perform label voting on points in the same voxel to determine voxel semantics, label voxels without sampling points as empty classes, establish a voxel semantic table, extract connected regions among voxels of the same class, and assign a unique instance number to each connected region. The voxel semantic table, instance number, and initial dataset are integrated together, and coordinate units, resolution, and semantic consistency are checked to generate a 3D semantic model, along with a version number and time stamp.

[0009] Optionally, the formation of the dynamic inspection virtual scene specifically includes: Read the 3D semantic model, import the voxel semantic table under a unified global coordinate and unit, set the simulation time step, spatial step and recording frequency, generate a scene initialization data package containing scene boundaries, voxel resolution and semantic mapping table, and complete the pre-allocation of scene rendering layer and physical layer resources. Taking the scene initialization data package as input, load environmental data including wind direction, wind speed, gust frequency, temperature, humidity and light intensity, establish an updatable data channel for hybrid driving of on-site sensing and historical database, generate a list of mobile obstacles according to building construction, set the initial position, speed range and behavior script respectively, determine the location and coverage parameters of communication base stations, and summarize to form an environmental parameter set and dynamic event plan table; Based on the environmental parameter set and dynamic event schedule, the performance boundaries and mission rules of the UAV are imported, including maximum speed, minimum turning radius, minimum safe distance, energy lower limit and take-off and landing constraints. Inspection target points are selected from the three-dimensional semantic model and assigned priority and time window. The execution order and re-inspection conditions are bound. The simulation environment is constructed according to the scenario initialization data package, environmental parameter set and mission rules to form a dynamic inspection virtual scenario.

[0010] Optionally, the formation of the multi-candidate path set specifically includes: The system reads the dynamic inspection virtual scene and inspection target points, initializes the reinforcement learning training environment, establishes the structural framework of the hierarchical reinforcement learning model, divides the model into an upper-level decision module and a lower-level control module, starts the upper-level decision and control module under a unified simulation time and coordinate system, groups the inspection target points according to spatial proximity and task priority, calculates the connectivity within the group and the distance between the groups in turn, arranges the task execution order according to the principle of minimum distance, generates a regional-level flight segment sequence, and allocates execution time and task duration to each flight segment. Based on the regional segment sequence, the lower-level control module is started. At the beginning of each segment, the scenario status interface is called to obtain the current position and heading data, the control parameters are updated at set intervals, the position of the next control point is calculated step by step on the calculation node, the direction of movement and speed are corrected according to the constraints of turning radius, climb angle and attitude angle, control commands are continuously recorded to form executable micro-track segments, and the end position of each track segment is used as the next starting point. All generated microtrack fragments are spliced ​​together according to time sequence, a continuity check is performed, paths with trajectory interruptions or deviations are smoothed and corrected, and an integrity verification is performed. At the same time, a unique path number is assigned to each path, and time matching, pose connection and motion reachability are verified. Paths that do not meet the continuity criteria are deleted and saved as a set of multiple candidate paths.

[0011] Optionally, the output of the route decision document specifically includes: Each candidate path in the multi-candidate path set is divided into segments according to adjacent path points. A parameterized ray path from the start point to the end point is established for each segment. Start and end time identifiers and length information are set for each segment. Ray sampling sequences are generated within the segment at set arc length intervals. The three-dimensional coordinates and segment number of each sampling point are recorded. For each sampling point, calculate the row, column, and layer indices of the voxel, and obtain the geometric center and boundary section identifier of the current voxel. Use affine perturbation hashing with alternating superposition of Morton codes to interleave and combine the binary bits of the three-axis index to obtain the Morton sequence code. Use a fixed odd number as scaling and offset to perform affine perturbation on the current Morton code and clip it to the predetermined width. At the same time, add the voxel index with three sets of distinct large odd prime numbers without carry to obtain the second integer digest, and then use another fixed odd number to amplify and clip it to the same width. XOR the two clipping results to obtain the generated voxel key value. For each segment of the ray sampling sequence, a three-axis synchronous voxel crossing determination is performed. Based on the sign of the ray direction component, the position of the voxel boundary above and below the three axes is determined. In each step, the distance to the lower interface in the three directions is calculated. The direction with the smallest distance is selected to advance one voxel layer, and the current position and voxel index are updated. Immediately after advancement, the voxel key value is used to query and mark the current voxel as traversable or blocked. At the same time, the current voxel key value, the entry and exit time, and the entry and exit surface identifiers are added to the segment-level crossing sequence in sequence until the end of the segment is reached or a blocking voxel appears. For each ray segment, a channel decision is made based on the crossing sequence. When all voxels in the sequence are marked as traversable, the current segment is recorded as legal. If any blocking voxel appears in the sequence, the current segment is recorded as illegal and subsequent decisions for that segment are stopped. For legal segments, the segment-level channel number is calculated according to the order of voxel key values ​​in the crossing sequence. The segment-level channel number uses a fixed base rolling superposition and takes the remainder under a preset modulus to ensure the irreversibility and uniqueness of the number. At the same time, the yaw angle and pitch angle are converted from the unit direction vector of the current segment, and the attitude window is determined within the allowable deviation range of the two. The distance window is determined by the values ​​corresponding to the minimum and maximum safe distances in the crossing sequence. The channel number, attitude window, distance window, and start and end time are recorded together in the segment result. All segment results are summarized into a channel registration table according to path number and segment sequence number. The registration content includes at least path number, segment sequence number, channel number, attitude window, distance window, start and end time, segment start and end coordinates, legality flag, and summary check code of segment-level voxel key value sequence. The channel registration table is checked for execution time continuity, channel number consistency and attitude boundary validity. Illegal or incomplete records are deleted and the route decision file is output.

[0012] Optionally, the process of forming the executable track set specifically includes: Read the route decision document and channel registration form, sort all track segments by path number and time order, remove channel segments marked as illegal, calculate the spatial distance between the end point and the start point of adjacent segments, and when the distance is less than a preset threshold, perform linear interpolation to generate a smooth transition point, merge the interpolation point with the original channel point to form a preliminary track set with continuous structure. The initial track set is subjected to channel masking. The attitude window and distance window parameters in the channel registration table are called. Window matching detection is performed on each track segment. When the track direction or distance deviation exceeds the window range, the current segment is marked as a masked segment and cut. The remaining track segments are reconstructed using a cubic spline function to generate a reconstructed track set. The reconstructed track set is subjected to continuous controllability verification. The rate of change of direction and rate of change of velocity are calculated segment by segment. The maximum climb angle, yaw rate and energy boundary are detected according to the UAV dynamic constraints. Track segments that do not meet the constraints are calculated in segments, the control commands are adjusted and controllable tracks are regenerated, and integrated to form a continuous and controllable executable track set.

[0013] Optionally, the generation of the task preparation file specifically includes: The executable track set is imported into the UAV control system. A task index table is established based on the path number. Waypoint sequences, attitude commands and track speed constraint parameters are loaded. A task initialization object is generated in the buffer area. Environmental detection is performed to complete the initialization of weather, magnetic field and communication status. The initialization task performs a route integrity check, comparing the coordinates and attitude sequences of adjacent waypoints segment by segment to confirm the path continuity and time sequence. If a breakpoint or overlapping segment is detected, a path repair operation is performed, and the expected energy requirement is obtained by integrating the track speed. The system compares the projected energy requirements with the current battery status of the drone. If the energy is insufficient, an energy warning record is generated. For missions that meet the requirements, the system integrates the track number, attitude command, and energy parameters to generate a mission preparation file, which is then stored in the cache.

[0014] Optionally, the process of outputting the task execution log specifically includes: Load waypoint sequence and attitude control parameters according to mission preparation file, initialize mission timer and flight status variables, complete attitude zeroing and sensor calibration, and enter automatic inspection mode after confirming starting point and heading. The flight control is executed according to the mission preparation document, the attitude and speed of the UAV are adjusted point by point, the position, attitude and energy data are collected in real time, and the trajectory deviation is detected. When the deviation exceeds the threshold, dynamic trajectory adjustment is performed to correct the direction and altitude of the next waypoint. Local obstacle avoidance replanning is triggered when obstacles or abnormal signals are detected. The system continuously records flight data, including waypoint coordinates, attitude changes, and energy consumption. After the mission is completed, it outputs a mission execution log, recording the mission number, track execution rate, and remaining energy. After integrity verification, the mission execution log is uploaded to the cloud.

[0015] Optionally, the implementation of the intelligent planning and adaptive closed-loop evolution specifically includes: The mission execution log is sorted by mission number and time tag. The waypoint sequence, attitude parameters, speed changes and energy consumption information are parsed. The actual flight trajectory is extracted and aligned with the planned trajectory in the mission preparation file. The spatial deviation value and time deviation between each waypoint are calculated, and a trajectory deviation dataset is generated. Perform statistical analysis on the track deviation dataset to calculate the average deviation, deviation direction distribution and cumulative error rate. Compare the deviation results with the historical execution logs stored in the cloud to identify error concentration areas and frequent attitude intervals. Generate a voxel index parameter adjustment table based on the analysis results to calibrate the key value perturbation factor and step interval parameters of the voxel hash ray spatial index. Using the parameter adjustment table as training input, the training sample set of the hierarchical reinforcement learning model is reconstructed. New state transition samples are generated based on the deviation sequence in the task execution log, which are used to retrain the upper-level decision-making strategy and the lower-level control strategy. The samples are then stored in the cloud to achieve intelligent planning and adaptive closed-loop evolution.

[0016] The beneficial effects of this invention are: 1. This invention utilizes the combination of a three-dimensional semantic model and a simulation environment to enable UAVs to complete task-level and action-level hierarchical reinforcement learning in virtual space. This allows for adaptive path decision-making and optimization in environments with complex building structures and dynamically changing obstacle distribution, significantly improving the flexibility and intelligence of trajectory planning.

[0017] 2. This invention introduces a voxel hash ray spatial indexing method to perform ray channel adjudication on candidate paths, enabling path determination to have higher spatial accuracy and real-time performance. This avoids the computational redundancy and decision delay problems in traditional voxel grid indexing, thereby ensuring the safety and continuity of UAV flight paths.

[0018] 3. This invention establishes a closed-loop optimization mechanism based on task execution logs, uploading execution data to the cloud for deviation analysis and strategy retraining, realizing the self-evolution and continuous optimization of the trajectory planning system, enabling the model to dynamically adjust voxel parameters and learning strategies according to historical tasks.

[0019] 4. This invention not only improves the inspection efficiency and energy utilization of UAVs in complex building environments, but also realizes intelligent closed-loop management of the entire process from path planning to task execution and strategy optimization. Attached Figure Description

[0020] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a flowchart of an intelligent planning method for unmanned aerial vehicle (UAV) inspection routes for buildings based on reinforcement learning, as proposed in this invention. Figure 2 This is a schematic diagram of the data structure for an intelligent planning method for building drone inspection routes based on reinforcement learning proposed in this invention. Detailed Implementation

[0021] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0022] refer to Figure 1-2 A method for intelligent planning of unmanned aerial vehicle (UAV) inspection routes for buildings based on reinforcement learning includes the following steps: Collect building facade outline information data, perform semantic segmentation on the building facade outline, generate a three-dimensional semantic model, and extract visible field constraints and risk area information at the same time; A simulation environment is built based on a three-dimensional semantic model to form an interactive, dynamic inspection virtual scene for subsequent strategy training and verification. In a dynamic inspection virtual scenario, a hierarchical reinforcement learning model is established. The upper-level strategy generates a building area-level flight segment sequence and time arrangement based on the semantic features of the inspection target point, while the lower-level strategy combines the environmental state to generate local micro-tracks and attitude control commands, forming a set of multiple candidate paths. A voxel hash ray spatial index is introduced into the multi-candidate path set. Ray channel decision is performed on each candidate track. A channel registration table containing channel number, attitude window and timestamp is constructed and the route decision file is output. Based on the route decision documents, the channel tracks are masked, and only the executable tracks confirmed by the index are retained, forming a continuous and controllable set of executable tracks; Import the set of executable flight paths into the UAV control system, initialize flight mission parameters, verify the integrity of the flight path and energy requirements, and generate mission preparation files. Perform flight inspection tasks according to the mission preparation documents, dynamically adjust the flight path and output the mission execution log; The task execution logs are uploaded to the cloud, and the trajectory deviations are analyzed and statistically analyzed. The results are used to calibrate the voxel hash ray spatial index parameters and retrain the strategy, so as to realize the intelligent planning system and adaptive closed-loop evolution.

[0023] This invention collects building facade outline information data and performs semantic segmentation on the building facade outline to generate a three-dimensional semantic model and extracts visible field constraints and risk area information, so that the geometric features and semantic features of the building are expressed in a unified manner, providing accurate spatial and semantic basic data for subsequent inspection path planning, and realizing automatic identification of inspection target points and determination of spatial constraints under complex building structures.

[0024] In this embodiment, the generation process of the three-dimensional semantic model specifically includes: Import building information files, read the geometric data and attribute information of building components, establish a global coordinate system, set a unified origin, direction and unit, transform the local coordinates of the building to global coordinates, extract the building facade outline data to obtain inspection target points, and resample the point cloud according to the set density. Determine the work space range according to the outer boundary, and divide it into a uniform cubic voxel grid according to the set voxel side length. Record the voxel number and spatial position parameters of each sampling point, and generate an initial dataset containing point cloud coordinates, voxel number and spatial boundary. Establish a semantic category set required for building inspection tasks, perform semantic segmentation on the point cloud of the initial dataset point by point, assign a unique category label to each point, aggregate the segmentation results by voxel number, perform label voting on points in the same voxel to determine voxel semantics, label voxels without sampling points as empty classes, establish a voxel semantic table, extract connected regions among voxels of the same class, and assign a unique instance number to each connected region. The voxel semantic table, instance number, and initial dataset are integrated together, and coordinate units, resolution, and semantic consistency are checked to generate a 3D semantic model, along with a version number and time stamp.

[0025] This invention constructs an interactive simulation environment based on a three-dimensional semantic model, which can simulate wind fields, obstacles and communication conditions in dynamic virtual scenes, providing an iterative training environment for reinforcement learning strategies, enabling the model to obtain highly adaptive planning capabilities under real constraints, and realizing highly consistent inspection task drills between virtual and reality.

[0026] In this embodiment, the formation of the dynamic inspection virtual scene specifically includes: Read the 3D semantic model, import the voxel semantic table under a unified global coordinate and unit, set the simulation time step, spatial step and recording frequency, generate a scene initialization data package containing scene boundaries, voxel resolution and semantic mapping table, and complete the pre-allocation of scene rendering layer and physical layer resources. Taking the scene initialization data package as input, load environmental data including wind direction, wind speed, gust frequency, temperature, humidity and light intensity, establish an updatable data channel for hybrid driving of on-site sensing and historical database, generate a list of mobile obstacles according to building construction, set the initial position, speed range and behavior script respectively, determine the location and coverage parameters of communication base stations, and summarize to form an environmental parameter set and dynamic event plan table; Based on the environmental parameter set and dynamic event schedule, the performance boundaries and mission rules of the UAV are imported, including maximum speed, minimum turning radius, minimum safe distance, energy lower limit and take-off and landing constraints. Inspection target points are selected from the three-dimensional semantic model and assigned priority and time window. The execution order and re-inspection conditions are bound. The simulation environment is constructed according to the scenario initialization data package, environmental parameter set and mission rules to form a dynamic inspection virtual scenario.

[0027] This invention establishes a hierarchical reinforcement learning model in a dynamic inspection virtual scenario. The upper-level strategy realizes regional segment planning and task time allocation, while the lower-level strategy generates local micro-tracks and attitude control commands. This enables hierarchical collaboration between global task planning and local path control, allowing UAVs to have global optimization and local dynamic response capabilities in complex building environments.

[0028] In this embodiment, the formation of the multiple candidate path set specifically includes: The system reads the dynamic inspection virtual scene and inspection target points, initializes the reinforcement learning training environment, establishes the structural framework of the hierarchical reinforcement learning model, divides the model into an upper-level decision module and a lower-level control module, starts the upper-level decision and control module under a unified simulation time and coordinate system, groups the inspection target points according to spatial proximity and task priority, calculates the connectivity within the group and the distance between the groups in turn, arranges the task execution order according to the principle of minimum distance, generates a regional-level flight segment sequence, and allocates execution time and task duration to each flight segment. Based on the regional segment sequence, the lower-level control module is started. At the beginning of each segment, the scenario status interface is called to obtain the current position and heading data, the control parameters are updated at set intervals, the position of the next control point is calculated step by step on the calculation node, the direction of movement and speed are corrected according to the constraints of turning radius, climb angle and attitude angle, control commands are continuously recorded to form executable micro-track segments, and the end position of each track segment is used as the next starting point. All generated microtrack fragments are spliced ​​together according to time sequence, a continuity check is performed, paths with trajectory interruptions or deviations are smoothed and corrected, and an integrity verification is performed. At the same time, a unique path number is assigned to each path, and time matching, pose connection and motion reachability are verified. Paths that do not meet the continuity criteria are deleted and saved as a set of multiple candidate paths.

[0029] This invention introduces a voxel hash ray spatial index into a multi-candidate path set, performs ray channel adjudication on each candidate track and constructs a channel registration table, which can efficiently identify passable and blocked areas in three-dimensional voxel space, realize real-time adjudication and accurate screening of tracks, and ensure that the generated routes have spatial continuity and safety.

[0030] In this embodiment, the output of the route decision document specifically includes: Each candidate path in the multi-candidate path set is divided into segments according to adjacent path points. A parameterized ray path from the start point to the end point is established for each segment. Start and end time identifiers and length information are set for each segment. Ray sampling sequences are generated within the segment at set arc length intervals. The three-dimensional coordinates and segment number of each sampling point are recorded. For each sampling point, calculate the row, column, and layer indices of the voxel, and obtain the geometric center and boundary section identifier of the current voxel. Using an affine perturbation hash with interleaved Morton codes, interleave the binary bits of the three-axis indices to obtain the Morton sequence code. Apply an affine perturbation to the current Morton code using a fixed odd number as scaling and offset, and crop it to the predetermined width. Simultaneously, add the voxel index to three distinct large odd prime numbers without carry to obtain the second integer digest, and then amplify it with another fixed odd number before cropping it to the same width. XOR the two cropping results bitwise to obtain the generated voxel key value. ; in, Indicates voxel bond value, These represent the three-dimensional integer indices of a voxel, namely row, column, and layer. This represents the Morton code obtained by interleaving three-axis bits. Let represent the affine scaling factor, affine translation offset, and amplification factor of the second branch of the Morton code, respectively. Indicates bitwise XOR, This indicates a no-carry addition operation. Represents distinct odd primes, used for... Integer weights for linear combinations. This represents the modulo operation. Indicates the hash bit width; For each segment of the ray sampling sequence, a three-axis synchronous voxel crossing determination is performed. Based on the sign of the ray direction component, the position of the voxel boundary above and below the three axes is determined. In each step, the distance to the lower interface in the three directions is calculated. The direction with the smallest distance is selected to advance one voxel layer, and the current position and voxel index are updated. Immediately after advancement, the voxel key value is used to query and mark the current voxel as traversable or blocked. At the same time, the current voxel key value, the entry and exit time, and the entry and exit surface identifiers are added to the segment-level crossing sequence in sequence until the end of the segment is reached or a blocking voxel appears. For each ray segment, a channel decision is made based on the crossing sequence. When all voxels in the sequence are marked as traversable, the current segment is recorded as legal. If any blocking voxel appears in the sequence, the current segment is recorded as illegal and subsequent decisions for that segment are stopped. For legal segments, the segment-level channel number is calculated according to the order of voxel key values ​​in the crossing sequence. The segment-level channel number uses a fixed base rolling superposition and takes the remainder under a preset modulus to ensure the irreversibility and uniqueness of the number. At the same time, the yaw angle and pitch angle are converted from the unit direction vector of the current segment, and the attitude window is determined within the allowable deviation range of the two. The distance window is determined by the values ​​corresponding to the minimum and maximum safe distances in the crossing sequence. The channel number, attitude window, distance window, and start and end time are recorded together in the segment result. All segment results are summarized into a channel registration table according to path number and segment sequence number. The registration content includes at least path number, segment sequence number, channel number, attitude window, distance window, start and end time, segment start and end coordinates, legality flag, and summary check code of segment-level voxel key value sequence. The channel registration table is checked for execution time continuity, channel number consistency and attitude boundary validity. Illegal or incomplete records are deleted and the route decision file is output.

[0031] This invention performs channel track masking based on route decision files, retaining only legal path segments verified by the index, and performs smoothing and dynamic constraint correction on the track to make the route continuous and controllable, avoiding flight instability caused by path incoherence or dynamic over-limit, and achieving high reliability output of UAV trajectory planning.

[0032] In this embodiment, the process of forming the executable track set specifically includes: Read the route decision document and channel registration form, sort all track segments by path number and time order, remove channel segments marked as illegal, calculate the spatial distance between the end point and the start point of adjacent segments, and when the distance is less than a preset threshold, perform linear interpolation to generate a smooth transition point, merge the interpolation point with the original channel point to form a preliminary track set with continuous structure. The initial track set is subjected to channel masking. The attitude window and distance window parameters in the channel registration table are called. Window matching detection is performed on each track segment. When the track direction or distance deviation exceeds the window range, the current segment is marked as a masked segment and cut. The remaining track segments are reconstructed using a cubic spline function to generate a reconstructed track set. The reconstructed track set is subjected to continuous controllability verification. The rate of change of direction and rate of change of velocity are calculated segment by segment. The maximum climb angle, yaw rate and energy boundary are detected according to the UAV dynamic constraints. Track segments that do not meet the constraints are calculated in segments, the control commands are adjusted and controllable tracks are regenerated, and integrated to form a continuous and controllable executable track set.

[0033] This invention imports a set of executable flight paths into the UAV control system and generates a mission preparation file based on mission parameters and energy boundary conditions. It can verify the integrity of the flight path, attitude sequence, and energy requirements before mission execution, thereby ensuring the reachability and energy safety of the flight mission during the execution phase and reducing the risk of mid-flight failure.

[0034] In this embodiment, the generation of the task preparation file specifically includes: The executable track set is imported into the UAV control system. A task index table is established based on the path number. Waypoint sequences, attitude commands and track speed constraint parameters are loaded. A task initialization object is generated in the buffer area. Environmental detection is performed to complete the initialization of weather, magnetic field and communication status. The initialization task performs a route integrity check, comparing the coordinates and attitude sequences of adjacent waypoints segment by segment to confirm the path continuity and time sequence. If a breakpoint or overlapping segment is detected, a path repair operation is performed, and the expected energy requirement is obtained by integrating the track speed. The system compares the projected energy requirements with the current battery status of the drone. If the energy is insufficient, an energy warning record is generated. For missions that meet the requirements, the system integrates the track number, attitude command, and energy parameters to generate a mission preparation file, which is then stored in the cache.

[0035] This invention executes flight inspection tasks according to the mission preparation document, and achieves immediate response to deviations through real-time attitude control and dynamic trajectory correction. At the same time, it automatically records mission execution logs and energy consumption information, enabling the UAV to have self-adjustment and self-recording functions in complex environments, providing a reliable basis for subsequent data analysis.

[0036] In this embodiment, the process of outputting the task execution log specifically includes: Load waypoint sequence and attitude control parameters according to mission preparation file, initialize mission timer and flight status variables, complete attitude zeroing and sensor calibration, and enter automatic inspection mode after confirming starting point and heading. The flight control is executed according to the mission preparation document, the attitude and speed of the UAV are adjusted point by point, the position, attitude and energy data are collected in real time, and the trajectory deviation is detected. When the deviation exceeds the threshold, dynamic trajectory adjustment is performed to correct the direction and altitude of the next waypoint. Local obstacle avoidance replanning is triggered when obstacles or abnormal signals are detected. The system continuously records flight data, including waypoint coordinates, attitude changes, and energy consumption. After the mission is completed, it outputs a mission execution log, recording the mission number, track execution rate, and remaining energy. After integrity verification, the mission execution log is uploaded to the cloud.

[0037] This invention achieves adaptive optimization of parameters and strategies by uploading task execution logs to the cloud and analyzing and statistically analyzing track deviations. The results are used to calibrate voxel hash ray spatial index parameters and retrain the hierarchical reinforcement learning model, enabling the system to continuously improve the accuracy and intelligence of route planning in multiple task iterations.

[0038] In this embodiment, the implementation of intelligent planning and adaptive closed-loop evolution specifically includes: The mission execution log is sorted by mission number and time tag. The waypoint sequence, attitude parameters, speed changes and energy consumption information are parsed. The actual flight trajectory is extracted and aligned with the planned trajectory in the mission preparation file. The spatial deviation value and time deviation between each waypoint are calculated, and a trajectory deviation dataset is generated. Perform statistical analysis on the track deviation dataset to calculate the average deviation, deviation direction distribution and cumulative error rate. Compare the deviation results with the historical execution logs stored in the cloud to identify error concentration areas and frequent attitude intervals. Generate a voxel index parameter adjustment table based on the analysis results to calibrate the key value perturbation factor and step interval parameters of the voxel hash ray spatial index. Using the parameter adjustment table as training input, the training sample set of the hierarchical reinforcement learning model is reconstructed. New state transition samples are generated based on the deviation sequence in the task execution log, which are used to retrain the upper-level decision-making strategy and the lower-level control strategy. The samples are then stored in the cloud to achieve intelligent planning and adaptive closed-loop evolution.

[0039] This invention realizes intelligent closed-loop management of the entire process of building drone inspection routes, from environmental modeling, virtual simulation, path planning to adaptive optimization. It can achieve efficient, safe, and self-learning route planning in complex building environments, effectively improving the automation level and long-term operational stability of drone inspection tasks.

[0040] Example 1: To verify the feasibility of this invention in practice, it was applied to the exterior facade inspection of a high-rise complex building. This building has a complex structure, including various curtain wall forms, curved facades, and rooftop platforms of different levels. Traditional UAV inspection methods require manual marking of inspection points and flight path planning, with an average planning time of approximately 3.5 hours. Furthermore, due to the building's complex geometry, the inspection trajectory exhibits significant blind spots and repetitive flight paths, resulting in an overall task time of approximately 120 minutes. To overcome these problems, this invention employs a joint planning method combining hierarchical reinforcement learning and voxel hash ray spatial indexing. Task pre-training and trajectory optimization are performed in a simulation environment to achieve intelligent, adaptive flight path planning and dynamic adjustment of the UAV.

[0041] In the implementation process, structural features are first analyzed using Building Information Modeling (BIM) data to generate a 3D semantic model containing semantic labels. This model finely segments the building facade, curtain wall, and roof areas, forming a semantic information set with spatial topological relationships. Based on this, comprehensive simulations of the building environment's lighting, wind field, temperature and humidity, and communication coverage are performed to give the virtual scene dynamic characteristics, providing realistic training feedback for the reinforcement learning model. The upper-level strategy is responsible for global task planning, automatically dividing inspection areas and generating flight segment execution sequences based on semantic features; the lower-level strategy controls the UAV's attitude and trajectory in local space, enabling it to flexibly adjust its attitude in areas with dense obstacles and narrow passages, avoiding collisions and repeated flights. Through this hierarchical training model, the UAV not only possesses global task planning capabilities but also achieves local adaptive flight decisions, thereby significantly improving inspection efficiency and path stability.

[0042] In the path generation and optimization process, this invention introduces a voxel hash ray spatial indexing method. This method achieves rapid determination of the drivability of track segments through efficient encoding and hashing of three-dimensional spatial voxels. The generation of voxel keys adopts a spatial mapping and affine perturbation mechanism, which enables track verification to maintain high computational stability and spatial resolution in complex structures. During execution, the system automatically filters out track segments determined to be blocked, retaining only paths with continuous controllability, and ensures the dynamic reachability of the trajectory under attitude and velocity constraints through interpolation and smoothing algorithms. After multiple rounds of filtering and reconstruction, the system generates a set of executable tracks and automatically imports them into the UAV control system to complete task parameter initialization and energy boundary checks. During the task execution phase, the UAV flies automatically according to the generated tracks and performs deviation detection and track adjustment based on the actual flight status. When the environment changes or communication signals are interfered with, the system will immediately trigger local obstacle avoidance and track replanning to achieve dynamic response and safe task execution.

[0043] Table 1. Performance Comparison Data Between the Method of the Invention and Traditional Methods

[0044] As shown in Table 1, the route planning time was reduced from 210 minutes to 59 minutes, improving efficiency by approximately 72%, the track repetition rate decreased by 61.3%, and the total inspection time was shortened by 24.2%, significantly improving task execution efficiency. The energy utilization rate increased from 81.0% to 93.4%, an increase of 15.3%, indicating that the present invention can allocate flight energy more efficiently. The track execution rate increased by 14.6%, and the average track deviation decreased by 67.9%, indicating that path generation is more accurate and stable. The index query speed increased by 2.8 times, and the number of model retraining convergence rounds decreased by 33.3%, demonstrating the algorithm's advantages in spatial indexing and learning efficiency.

[0045] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A method for intelligent planning of unmanned aerial vehicle (UAV) inspection routes for buildings based on reinforcement learning, characterized in that, Includes the following steps: Collect building facade outline information data, perform semantic segmentation on the building facade outline, and generate a three-dimensional semantic model; A simulation environment is constructed based on a three-dimensional semantic model to form a dynamic inspection virtual scene. In a dynamic inspection virtual scenario, a hierarchical reinforcement learning model is established. The upper-level strategy combines the semantic features of the inspection target point, and the lower-level strategy combines the environmental state to form a set of multiple candidate paths. Introduce a voxel hash ray spatial index into the multi-candidate path set, perform ray channel decision for each candidate track, construct a channel registration table, and output the route decision file; Based on the route decision documents, the channel tracks are masked, and a continuous and controllable set of executable tracks is formed. Import the set of executable flight paths into the UAV control system, initialize flight mission parameters, verify the integrity of the flight path and energy requirements, and generate mission preparation files. Perform flight inspection tasks according to the mission preparation documents, dynamically adjust the flight path and output the mission execution log; The mission execution logs are uploaded to the cloud, and the trajectory deviations are analyzed and statistically analyzed. The results are used to calibrate the voxel hash ray spatial index parameters and retrain the strategy, so as to realize intelligent planning and adaptive closed-loop evolution.

2. The intelligent planning method for unmanned aerial vehicle (UAV) inspection routes for buildings based on reinforcement learning according to claim 1, characterized in that, The process of generating the three-dimensional semantic model specifically includes: Import building information files, read the geometric data and attribute information of building components, establish a global coordinate system, extract the building facade contour data to obtain inspection target points, and resample the point cloud according to the set density to generate the initial dataset; Establish a semantic category set required for building inspection tasks, perform semantic segmentation on the point cloud of the initial dataset point by point, assign a unique category label to each point, aggregate the segmentation results by voxel number, establish a voxel semantic table, and assign a unique instance number to each connected region. The voxel semantic table, instance numbers, and initial dataset are integrated together, and a semantic consistency check is performed to generate a three-dimensional semantic model.

3. The intelligent planning method for unmanned aerial vehicle (UAV) inspection routes for buildings based on reinforcement learning according to claim 1, characterized in that, The formation of the dynamic inspection virtual scene specifically includes: Read the 3D semantic model, import the voxel semantic table under a unified global coordinate and unit, generate the scene initialization data package, and complete the pre-allocation of the scene's rendering layer and physical layer resources. Using the scene initialization data package as input, load environmental data, establish data channels, generate a list of mobile obstacles, and form an environmental parameter set and a dynamic event schedule. Based on the environmental parameter set and dynamic event schedule, the performance boundaries and task rules of the UAV are imported. Inspection target points are selected from the 3D semantic model and priority and time window are assigned. The simulation environment is constructed according to the scenario initialization data package, environmental parameter set and task rules to form a dynamic inspection virtual scenario.

4. The intelligent planning method for unmanned aerial vehicle (UAV) inspection routes for buildings based on reinforcement learning according to claim 1, characterized in that, The formation of the multi-candidate path set specifically includes: Read the dynamic inspection virtual scene and inspection target points, initialize the reinforcement learning training environment, establish the structural framework of the hierarchical reinforcement learning model, divide the model into an upper-level decision module and a lower-level control module, start the upper-level decision and control module under a unified simulation time and coordinate system, group the inspection target points according to spatial proximity and operation priority, calculate the connectivity within the group and the distance between the groups in turn, arrange the task execution order according to the principle of minimum distance, and generate a regional-level flight segment sequence. Based on the regional segment sequence, the lower-level control module is started. At the beginning of each segment, the scene status interface is called to obtain the current position and heading data, the control parameters are updated at set intervals, the position of the next control point is calculated step by step on the calculation node, the movement direction and speed are corrected, the control commands are continuously recorded, micro-track segments are formed, and the end position of each track segment is used as the next starting point. All generated microtrack fragments are spliced ​​together according to time sequence, a continuity check is performed, paths with track interruptions or deviations are smoothed and corrected, and an integrity verification is performed. At the same time, a unique path number is assigned to each path and saved as a set of multiple candidate paths.

5. The intelligent planning method for unmanned aerial vehicle (UAV) inspection routes for buildings based on reinforcement learning according to claim 1, characterized in that, The output of the route decision document specifically includes: Each candidate path in the multi-candidate path set is divided into segments according to adjacent path points. A parameterized ray path from the start point to the end point is established for each segment. Start and end time identifiers and length information are set for each segment. Ray sampling sequences are generated within the segment at set arc length intervals. The three-dimensional coordinates and segment number of each sampling point are recorded. For each sampling point, calculate the row, column, and layer indices of the voxel, and obtain the geometric center and boundary section identifier of the current voxel. Use affine perturbation hashing with alternating superposition of Morton codes to interleave and combine the binary bits of the three-axis index to obtain the Morton sequence code. Use a fixed odd number as scaling and offset to perform affine perturbation on the current Morton code and clip it to the predetermined width. At the same time, add the voxel index with three sets of distinct large odd prime numbers without carry to obtain the second integer digest, and then use another fixed odd number to amplify and clip it to the same width. XOR the two clipping results to obtain the generated voxel key value. For each segment of the ray sampling sequence, a three-axis synchronous voxel crossing determination is performed. Based on the sign of the ray direction component, the position of the voxel boundary above and below the three axes is determined. In each step, the distance to the lower interface in the three directions is calculated. The direction with the smallest distance is selected to advance a voxel layer. Immediately after advancement, the voxel key value is used to query and mark the current voxel as crossable or blocked. At the same time, the current voxel key value, the entry and exit time, and the entry and exit surface identifiers are added to the segment-level crossing sequence in sequence. For each ray segment, a channel decision is made based on the crossing sequence. When all voxels in the sequence are marked as traversable, the current segment is recorded as legal. For legal segments, the segment-level channel number is calculated according to the order of voxel key values ​​in the crossing sequence. At the same time, the yaw angle and pitch angle are converted from the unit direction vector of the current segment, and the attitude window is determined within the allowable deviation range of the two. The distance window is determined by the values ​​corresponding to the minimum and maximum safe distances in the crossing sequence. The channel number, attitude window, distance window, and start and end time are recorded together in the segment result. All segment results are summarized into a channel registration table according to path number and segment sequence number. The continuity of execution time, consistency of channel number and validity of attitude boundary are verified in the channel registration table, and the route decision document is output.

6. The intelligent planning method for unmanned aerial vehicle (UAV) inspection routes for buildings based on reinforcement learning according to claim 1, characterized in that, The process of forming the executable track set specifically includes: Read the route decision document and channel registration form, sort all track segments according to path number and time order, calculate the spatial distance between the end point and the start point of adjacent segments, and when the distance is less than the preset threshold, perform linear interpolation, merge the interpolation point with the original channel point, and form a preliminary track set with continuous structure. The initial track set is subjected to channel masking. The attitude window and distance window parameters in the channel registration table are called to perform window matching detection on each track segment and generate a reconstructed track set. Perform a continuous controllability check on the reconstructed track set, perform segmented calculations on track segments that do not meet the constraints, adjust the control commands and regenerate controllable tracks, and integrate them to form a continuous and controllable executable track set.

7. The intelligent planning method for unmanned aerial vehicle (UAV) inspection routes for buildings based on reinforcement learning according to claim 1, characterized in that, The generation of the task preparation file specifically includes: Import the set of executable tracks into the UAV control system, establish a task index table based on the path number, and generate a task initialization object in the cache area; Perform route integrity verification on the initialization task, compare the coordinates and attitude sequences of adjacent waypoints segment by segment to confirm the path continuity and time sequence, and obtain the expected energy requirements; The projected energy requirements are compared with the current battery status of the drone, and the track number, attitude commands, and energy parameters are integrated to generate a mission preparation file.

8. The intelligent planning method for unmanned aerial vehicle (UAV) inspection routes for buildings based on reinforcement learning according to claim 1, characterized in that, The process of outputting the task execution log specifically includes: Load waypoint sequence and attitude control parameters according to mission preparation file, initialize mission timer and flight status variables, complete attitude zeroing and sensor calibration, and enter automatic inspection mode after confirming starting point and heading. Perform flight control according to the mission preparation document, adjust the UAV's attitude and speed point by point, collect position, attitude and energy data in real time, and detect track deviations; The system continuously records flight data and outputs a mission execution log after the mission is completed. After integrity verification, the mission execution log is uploaded to the cloud.

9. The intelligent planning method for unmanned aerial vehicle (UAV) inspection routes for buildings based on reinforcement learning according to claim 1, characterized in that, The implementation of intelligent planning and adaptive closed-loop evolution specifically includes: The mission execution log is sorted by mission number and time tag. The waypoint sequence, attitude parameters, speed changes and energy consumption information are parsed. The actual flight trajectory is extracted and aligned with the planned trajectory in the mission preparation file, and a trajectory deviation dataset is generated. Perform statistical analysis on the track deviation dataset, and generate a voxel index parameter adjustment table based on the analysis results. This table is used to calibrate the key value perturbation factor and step interval parameter of the voxel hash ray spatial index. Using the parameter adjustment table as training input, the training sample set of the hierarchical reinforcement learning model is reconstructed. New state transition samples are generated based on the deviation sequence in the task execution log and stored in the cloud to achieve intelligent planning and adaptive closed-loop evolution.

Images