Global inspection method and system for flying robot based on bim prior semantic map

Abstract

The present application relates to the technical field of intelligent inspection and autonomous navigation of flying robots in construction engineering, and particularly relates to a flying robot global inspection method and system based on a BIM prior semantic map, comprising: analyzing BIM model component attributes and voxelizing, constructing a multi-layer semantic map containing static occupancy, dynamic predicted occupancy, clearance risk, topology avoidance and target visibility layer; extracting safe free space according to adaptive inflation of obstacles by robot collision envelope; discretizing component surface into target patches, generating candidate viewpoints and establishing coverage matrix, and using coverage debt to represent inspection state; extracting cross-domain portals to form a unified task graph, issuing reversible corridor certificates for candidate trajectory prefix nodes, and solving global viewpoint access sequence; updating pose in online joint registration, performing multiple hypothesis testing, and correcting trajectory in the certificate pipeline; verifying that coverage debt is zero and the certificate is valid, and then exiting, and outputting inspection data bound to component identifiers.

Description

Technical Field

[0001] This application relates to the fields of intelligent inspection of building engineering and autonomous navigation of flying robots, and in particular to a method and system for full-domain inspection of flying robots based on BIM prior semantic maps. Background Technology

[0002] High-rise building scaffolding, attached lifting scaffolding, cantilevered scaffolding, and formwork support systems are characterized by their high altitude, narrow spaces, dense component density, and dispersed work surfaces. Traditional manual inspections struggle to simultaneously access internal structural members, wall ties, the bottom of cantilevered structures, and the back side of the facade. Furthermore, entering these areas increases safety risks such as falls from heights, collisions, and entrapment. Existing technologies have disclosed solutions for generating 2D navigation maps or model point clouds using BIM, semantically segmenting the site point cloud for autonomous robot navigation, and planning routes based on shortest distance and path traversability while correcting position online.

[0003] However, the aforementioned existing technologies have the following shortcomings: First, BIM is often simplified to static geometry or waypoint sources. Component semantics, risks, dynamic occupancy, clearance, target visibility, and exit costs are not uniformly queried by the planner, resulting in planning results that cannot be directly linked to specific components and target surfaces. Second, existing solutions only determine whether the robot has passed through a certain location, without verifying the effective coverage of the target surface in the image or point cloud. This leads to the problem of missing wall ties, cantilever bottoms, or backside nodes even after the route has been executed. Third, conventional local obstacle avoidance does not verify the reverse exit path before the robot enters a narrow passage, nor does it continuously compare the exit cost with the remaining energy. There are several potential issues: First, robots may be unable to safely exit narrow areas. Second, the internal structure, external building structure, and cantilevered bottom are often planned separately, lacking a unified task map across areas, leading to frequent manual intervention and difficulty in archiving inspection data. Third, the addition of personnel, hoisting equipment, and temporary material storage on site may disrupt the original observation posture or obstruct the target when bypassing obstacles. Fourth, existing solutions typically do not write back coverage loss to the task layer or automatically generate supplementary viewpoints. Fifth, there may be additions, omissions, or positional discrepancies between the design model and the actual scene. Relying solely on offline BIM will result in outdated maps, while relying solely on online mapping will lead to the loss of component IDs, categories, and design risk information. Summary of the Invention

[0004] To address the technical problems of existing BIM navigation technologies, such as the inability to verify the effective coverage quality of target component surfaces, the lack of verification of trajectory prefix reversibility before in-depth exploration, the lack of a unified expression of component semantics and planning status, and the absence of a unified cross-domain task map for multi-area inspection tasks, leading to missed inspections, getting stuck, and difficulties in unified data archiving, this invention aims to provide a full-domain inspection method for flying robots based on BIM prior semantic maps. The specific technical solution adopted is as follows:

[0005] Step S1: Obtain the building information model, inspection task, robot parameters, sensor parameters, safety exits and endurance constraints of the target project, complete the model version and coordinate system verification, analyze the geometry, category, risk and inspection attributes of each component, identify static obstacles, no-fly zones and target surfaces, perform multi-resolution voxelization on the building information model, and construct a multi-layer semantic map including a static occupancy layer, a dynamic prediction occupancy layer, a clearance risk layer, a topology avoidance layer and a target visibility layer;

[0006] Step S2: Obtain the collision envelope of the flying robot, determine the expansion amount based on the equivalent size of the body, the basic safety margin, the semantic risk additional margin, and the positioning uncertainty additional margin, perform adaptive expansion on the obstacle, calculate the effective clearance at any position, and extract the safe free space.

[0007] Step S3: Discretize the surface of the component to be inspected into target patches, generate candidate viewpoints based on the target patch normal, sensor field of view, working distance and occlusion relationship, establish a prediction coverage matrix offline, establish an evidence matrix online, use the coverage debt of the target patch to represent the inspection completion status, and automatically trigger supplementary testing when the coverage debt increases.

[0008] Step S4: Divide the task space into the internal area of ​​the frame, the external area of ​​the building, the cantilever bottom area and the high-risk area. Extract cross-domain portals at the boundaries of the areas and record the entry direction, attitude range, minimum clearance and available exit direction. Combine the areas into a unified task map through the cross-domain portals.

[0009] Step S5: Calculate the spatiotemporal reverse emergency pipeline for each prefix node of the candidate trajectory, verify the effective clearance constraint, dynamic prediction occupancy probability constraint, and exit energy consumption upper bound constraint, issue a reversible corridor certificate with map version and effective time window for nodes that satisfy all constraints, and immediately revoke the certificate and trigger the exit priority state machine when any constraint is exceeded. On the basis of valid certificate, solve the global viewpoint access sequence, main safe corridor and emergency exit tree at the cost of coverage benefit, path length, semantic risk, energy consumption and positioning uncertainty.

[0010] Step S6: Jointly register the airborne observations with the prior of the building information model and update the pose covariance. Perform five types of multi-hypothesis tests on the abnormal registration residuals in parallel: positioning drift, model bias, new dynamic obstacles, missing or displaced components, and perceived noise. Update the pose, dynamic layer, or model bias layer according to the posterior confidence level. Classify and track dynamic obstacles and generate time slice prediction occupancy envelopes. Correct the trajectory in the forward pipeline defined by the reversible corridor certificate and verify the reverse emergency pipeline node by node.

[0011] Step S7: Update the evidence matrix and coverage debt with actual pose, effective observation data, incident angle, occlusion rate and repeatability consistency. For target patches with increased coverage debt, generate alternative viewpoints within the valid area of ​​the reversible corridor certificate and perform supplementary measurements. After all target patches have zero coverage debt and the reversible corridor certificate is valid, perform a safe exit and output component-level inspection data and map increments bound to the component's globally unique identifier.

[0012] Preferably, step S1 includes:

[0013] Step S11: Obtain the building information model, inspection task list, robot parameters, sensor parameters, safety exit location and endurance constraints of the target project at the current construction stage; complete the model version and coordinate system verification; and confirm that there are no abnormal scales, duplicate components or missing components in the model.

[0014] Step S12: Analyze the geometry, category, risk level and inspection attributes of each component in the building information model, identify static obstacles, no-fly zones and target surfaces, and establish a unified index for each component that includes geometric area, category, risk level, inspection attributes and version information. The component semantics come from IFC entity type, family name, component globally unique identifier or attribute field.

[0015] Step S13: Perform multi-resolution voxelization on the building information model. Use fine-resolution voxels in areas with dense components, concentrated targets, or large clearance gradients, and use coarse-resolution voxels in open areas. Connect areas of different resolutions through octrees, sparse voxels, or multi-scale topology graphs.

[0016] Step S14: Construct a multi-layer semantic map that includes a static occupancy layer, a dynamic prediction occupancy layer, a clearance risk layer, a topology retreat layer, and a target visibility layer. Each layer can be queried uniformly through a globally unique identifier for the component, a spatial index, and a timestamp.

[0017] Preferably, step S2 includes:

[0018] Step S21: Obtain the collision envelope of the flying robot, wherein the collision envelope is approximated by one of the following: circumscribed ellipsoid, circumscribed cylinder, or a combination of multiple spheres;

[0019] Step S22: Determine the obstacle expansion amount, which is jointly determined by the equivalent size of the body, the basic safety margin, the semantic risk additional margin, and the positioning uncertainty additional margin;

[0020] Step S23: Calculate the effective clearance at any location by deducting the expansion amount of the equivalent radius of the aircraft, the basic safety margin, the semantic risk margin, and the positioning uncertainty margin from the distance to the obstacle;

[0021] Step S24: Voxels with effective clearance not greater than zero are identified as inaccessible areas, and the remaining voxel space with effective clearance greater than zero after expansion constitutes a safe free space.

[0022] Preferably, step S3 includes:

[0023] Step S31: Discretize the surface of the component to be inspected into target patches according to area, curvature, risk level and detection resolution, and generate one or more candidate viewpoints for each target patch within the effective working distance of the sensor based on its surface normal.

[0024] Step S32: Establish a prediction coverage matrix offline, where each element represents the expected coverage quality of the target patch by the candidate viewpoint; establish an evidence matrix online, whose elements are calculated from the actual pose, effective pixels or points, sharpness, incident angle, occlusion rate and consistency of repeated observations.

[0025] Step S33: Define the target patch coverage debt as the non-negative value of the difference between the number of independent valid observations required for the target patch and the actual number of observations in the evidence matrix that reach the observation quality threshold. When obstacle avoidance, attitude deviation or new occlusion causes the evidence matrix to decrease, the coverage debt will automatically increase again. High-risk patches require independent evidence from different view clusters.

[0026] Preferably, step S4 includes:

[0027] Step S41: Based on component type, clearance distribution, spatial connectivity, and target distribution, divide the task space into the internal area of ​​the frame, the external area of ​​the building, the cantilever bottom area, and the high-risk area. Set map resolution, speed limit, safety margin, and viewpoint generation strategy for each area.

[0028] Step S42: Extract cross-domain portal nodes at the region boundary and record the entry direction, attitude range, minimum clearance, nearby targets, regional risk level, and available exit direction for each portal;

[0029] Step S43: Connect the internal area of ​​the frame, the external area of ​​the building, and the cantilevered bottom area into a unified task map through a cross-domain portal, eliminating the need for manual takeover and realizing unified archiving of inspection data.

[0030] Preferably, step S5 includes:

[0031] Step S51: Calculate the spatiotemporal reverse emergency pipeline to the safe exit for each prefix node of the candidate trajectory. All states in the reverse emergency pipeline must meet the following conditions: the dynamic prediction occupancy probability is lower than the threshold, the effective clearance is greater than zero, and the flight control dynamics constraints are met.

[0032] Step S52: Verify three constraints: Effective clearance constraint requires that the effective clearance of all predicted states in the reverse emergency pipeline is greater than zero; Dynamic occupancy probability constraint requires that the predicted collision probability of all states in the pipeline is lower than the threshold; Energy reserve constraint requires that the sum of the exit energy consumption upper limit, return reserve energy consumption and uncertainty additional energy consumption does not exceed the current remaining energy. The exit energy consumption upper limit is estimated based on the exit path length, climb altitude, wind disturbance additional amount and hovering waiting time.

[0033] Step S53: Issue a reversible corridor certificate to the prefix node that satisfies all constraints. The certificate includes the trajectory version, map version, effective time window, safety exit identifier, minimum effective clearance, maximum predicted collision probability, and upper limit of exit energy consumption. The certificate is immediately revoked when any field exceeds the limit or the map version changes, and the state machine switches to the execution sequence of stopping exploration, retreating to a safe node, restoring the certificate, and replanning.

[0034] Step S54: Based on the validity of the reversible corridor certificate, construct an attributed topology graph from candidate viewpoints and cross-domain portals, and solve the global viewpoint access sequence, main security corridor and emergency exit tree at the cost of coverage benefits, path length, semantic risk, energy consumption, area switching cost and positioning uncertainty.

[0035] Preferably, step S6 includes:

[0036] Step S61: Jointly register the pose estimation with covariance formed by airborne vision, lidar, IMU, barometer and ranging sensor with the prior of building information model, and update the pose covariance;

[0037] Step S62: Parallel maintenance of the abnormal registration residuals for the positioning drift hypothesis, model version deviation hypothesis, newly added dynamic obstacle hypothesis, design component missing or displacement hypothesis, and perception noise hypothesis. Calculate the posterior probability of each hypothesis based on the spatial consistency, duration, whether it moves with the robot, whether it is consistent with the component boundary, and multi-view repeatability of the residuals. Only when the posterior probability of a certain hypothesis exceeds the confidence threshold and the difference between it and the posterior probability of the suboptimal hypothesis reaches the confidence interval, the corresponding update is performed.

[0038] Step S63: Classify and track dynamic obstacles, generate time-slice predicted occupancy envelopes based on velocity covariance, and update the dynamic predicted occupancy layer in real time.

[0039] Step S64: Correct the trajectory within the forward pipeline defined by the reversible corridor certificate, and check the reverse emergency pipeline node by node. If a dynamic obstacle blocks the exit path, the state machine immediately stops going deeper and retreats to the nearest safe node. When the certificate expires, stop going deeper and prioritize restoring reversibility.

[0040] Preferably, step S7 includes:

[0041] Step S71: Update the evidence matrix and coverage debt with actual pose, effective target pixels or points, occlusion ratio, incident angle, sharpness and repeat observation consistency;

[0042] Step S72: For the target patch covering the debt rebound, prioritize generating alternative viewpoints within the effective area of ​​the reversible corridor certificate and perform supplementary testing. If there is no solution locally, reselect the portal or adjust the global task order.

[0043] Step S73: If the reversible corridor certificate has expired, perform a safe exit first, retreat to a safe node or hovering area, and then perform a global replanning. Do not replace the conclusion that the check is completed with the conclusion that the path has flown.

[0044] Step S74: After all key target patches are covered, debt is cleared, and the current location reversible corridor certificate is valid, perform a safe exit or return, output component-level inspection data bound to the component's globally unique identifier, record obstacle avoidance trajectory, certificate failure reason, compensation result, and map increment, and archive.

[0045] To address the aforementioned technical problems, this invention also provides: a full-domain inspection system for flying robots based on BIM prior semantic maps, the system comprising:

[0046] The BIM model acquisition and verification module is used to: acquire the building information model, inspection tasks, robot parameters, sensor parameters, safety exits and endurance constraints of the target project, and complete the model version and coordinate system verification.

[0047] The component semantic parsing module is used to: parse the geometry, category, risk and inspection attributes of each component, and identify static obstacles, no-fly zones and target surfaces;

[0048] The multi-layer semantic map construction module is used to: perform multi-resolution voxelization on the building information model to construct a multi-layer semantic map including a static occupancy layer, a dynamic predicted occupancy layer, a clearance risk layer, a topology avoidance layer, and a target visibility layer; obtain the collision envelope of the flying robot, determine the expansion amount based on the equivalent size of the body, the basic safety margin, the semantic risk additional margin, and the positioning uncertainty additional margin, perform adaptive expansion on obstacles, calculate the effective clearance at any position, and extract the safe free space;

[0049] The target surface and viewpoint generation module is used to: discretize the surface of the component to be inspected into target patches, generate candidate viewpoints based on the target patch normal, sensor field of view, working distance and occlusion relationship, establish a prediction coverage matrix offline, establish an evidence matrix online, use the target patch coverage debt to represent the inspection completion status, and automatically trigger supplementary testing when the coverage debt increases.

[0050] The bidirectional reachability and exit verification module is used to: calculate the spatiotemporal reverse emergency pipeline for each prefix node of the candidate trajectory, verify the effective clearance constraint, the dynamic prediction occupancy probability constraint and the exit energy consumption upper limit constraint, issue a reversible corridor certificate with map version and effective time window for nodes that meet all constraints, and immediately revoke the certificate and trigger the exit priority state machine when any constraint is exceeded.

[0051] The security corridor and global planning module is used to: solve for the global viewpoint access sequence, main security corridor, and emergency exit tree at the cost of coverage benefits, path length, semantic risks, energy consumption, and positioning uncertainty, based on the validity of the certificate;

[0052] The real-scene registration and fusion positioning module is used to: jointly register airborne observations with the prior of the building information model and update the pose covariance; perform five types of multi-hypothesis tests on abnormal registration residuals in parallel, including positioning drift, model bias, new dynamic obstacles, missing or displaced components, and perceived noise; and update the pose, dynamic layer or model bias layer according to the posterior confidence.

[0053] The dynamic semantic perception module is used to classify and track dynamic obstacles and generate time-slice predicted occupancy envelopes.

[0054] The local trajectory correction module is used to: correct the trajectory within the forward pipeline defined by the reversible corridor certificate and verify the reverse emergency pipeline node by node;

[0055] The coverage verification and data management module is used to: update the evidence matrix and coverage debt with actual pose, effective observation data, incident angle, occlusion rate and repeatability consistency; generate alternative viewpoints and perform supplementary measurements for target patches with increased coverage debt within the effective area of ​​the reversible corridor certificate; perform safe exit after all target patches have zero coverage debt and the reversible corridor certificate is valid; and output component-level inspection data and map increments bound to the component's globally unique identifier.

[0056] The present invention has the following beneficial effects:

[0057] 1. First, BIM is upgraded from a geometric background to an executable semantic state. Component category, risk, occupancy, clearance, visibility, and exit cost are associated in the same multi-layer semantic map. Planning results can be directly traced to specific components and target surfaces, solving the problem of the disconnect between component semantics and planning state in existing solutions. Then, adaptive expansion of obstacles is performed based on the robot's actual collision envelope and positioning uncertainty. Effective clearance calculation considers robot size, safety margin, semantic risk, and positioning covariance, avoiding overestimation of passable clearance by point models or coarse grids, thus improving the safety of narrow space paths. Secondly, a coverage debt mechanism forms a closed loop between target patches, candidate viewpoints, evidence matrix, and actual observation quality. This can identify missed detections caused by obstacle avoidance or occlusion and automatically generate alternative viewpoints for supplementary measurements, transforming "flying over" into "detection," solving the problem that existing solutions cannot verify the effective coverage quality of target surfaces. Next, a reversible corridor certificate is issued for each prefix node of the candidate trajectory. The entry path and exit path are verified in pairs. At each key node, the exit cost, clearance and remaining energy are continuously compared. When the certificate expires, the exit capability is restored first, which explicitly prevents the robot from getting stuck in narrow areas.

[0058] Then, the cross-domain portal connects the interior of the framework, the exterior of the building, and the underside of the cantilever into a unified task map, enabling integrated multi-area inspection, eliminating the need for manual intervention, and archiving inspection data uniformly using globally unique component identifiers as indexes. Furthermore, the multi-hypothesis testing mechanism allows the system to utilize BIM's global structure and component identifiers while promptly reflecting new obstacles, missing components, or model deviations, taking into account both design priors and on-site changes. Finally, it reduces the necessity for personnel to enter high-altitude, narrow, and cantilevered areas, thereby reducing hazard exposure. This effect is a risk reduction, not an absolute guarantee of project safety. Each module can be distributed across onboard computing units, ground stations, or edge servers; map structures, solvers, and sensors are all replaceable, facilitating expansion to other complex spaces and multi-robot tasks with prior 3D design models, such as bridges, tunnels, factories, and storage tanks.

[0059] 2. The full-domain inspection system for flying robots based on BIM prior semantic maps provided by this invention has the same beneficial effects as the full-domain inspection method for flying robots based on BIM prior semantic maps provided by this invention, and will not be described in detail here. Attached Figure Description

[0060] To more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0061] Figure 1 This is a schematic diagram of the overall architecture of a BIM-based prior semantic map-based full-domain inspection method for flying robots provided in an embodiment of the present invention.

[0062] Figure 2 This is a schematic diagram of the multi-layer, multi-resolution BIM prior semantic map layered structure of a BIM prior semantic map-based full-domain inspection method for flying robots provided in an embodiment of the present invention.

[0063] Figure 3 This is a schematic diagram of the full-domain inspection method for flying robots based on BIM prior semantic maps, provided in an embodiment of the present invention.

[0064] Figure 4 This is a schematic diagram of the high-precision semantic navigation technology driven by BIM prior driving, which is an embodiment of the flying robot full-domain inspection method based on BIM prior semantic map provided by an embodiment of the present invention.

[0065] Figure 5 This is a schematic diagram of the internal-external integrated inspection and exit verification of the flying robot full-domain inspection method based on BIM prior semantic map provided in an embodiment of the present invention.

[0066] In the picture:

[0067] 100. Building structure; 200. External scaffolding or temporary facilities; 300. Flying inspection robot; 400. Inspection corridor inside the scaffolding; 500. Inspection corridor outside the building; 600. Emergency exit or return corridor; 700. Inspection target surface or connection node; 800. Temporary or dynamic obstacles. Detailed Implementation

[0068] To further illustrate the technical means and effects adopted by the present invention to achieve its intended purpose, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features, and effects of a BIM-based prior semantic map-based full-domain inspection method and system for flying robots proposed according to the present invention. In the following description, different "one embodiment" or "another embodiment" do not necessarily refer to the same embodiment. Furthermore, specific features, structures, or characteristics in one or more embodiments can be combined in any suitable form.

[0069] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains.

[0070] The following description, in conjunction with the accompanying drawings, details the specific scheme of the flying robot full-domain inspection method and system based on BIM prior semantic map provided by this invention.

[0071] Please combine Figures 1-5 The first embodiment of the present invention provides a method for full-domain inspection of a flying robot based on a BIM prior semantic map, the method comprising:

[0072] Step S1: Obtain the building information model, inspection task, robot parameters, sensor parameters, safety exits and endurance constraints of the target project, complete the model version and coordinate system verification, analyze the geometry, category, risk and inspection attributes of each component, identify static obstacles, no-fly zones and target surfaces, perform multi-resolution voxelization on the building information model, and construct a multi-layer semantic map including a static occupancy layer, a dynamic prediction occupancy layer, a clearance risk layer, a topology avoidance layer and a target visibility layer;

[0073] Step S2: Obtain the collision envelope of the flying robot, determine the expansion amount based on the equivalent size of the body, the basic safety margin, the semantic risk additional margin, and the positioning uncertainty additional margin, perform adaptive expansion on the obstacle, calculate the effective clearance at any position, and extract the safe free space.

[0074] Step S3: Discretize the surface of the component to be inspected into target patches, generate candidate viewpoints based on the target patch normal, sensor field of view, working distance and occlusion relationship, establish a prediction coverage matrix offline, establish an evidence matrix online, use the coverage debt of the target patch to represent the inspection completion status, and automatically trigger supplementary testing when the coverage debt increases.

[0075] Step S4: Divide the task space into the internal area of ​​the frame, the external area of ​​the building, the cantilever bottom area and the high-risk area. Extract cross-domain portals at the boundaries of the areas and record the entry direction, attitude range, minimum clearance and available exit direction. Combine the areas into a unified task map through the cross-domain portals.

[0076] Step S5: Calculate the spatiotemporal reverse emergency pipeline for each prefix node of the candidate trajectory, verify the effective clearance constraint, dynamic prediction occupancy probability constraint, and exit energy consumption upper bound constraint, issue a reversible corridor certificate with map version and effective time window for nodes that satisfy all constraints, and immediately revoke the certificate and trigger the exit priority state machine when any constraint is exceeded. On the basis of valid certificate, solve the global viewpoint access sequence, main safe corridor and emergency exit tree at the cost of coverage benefit, path length, semantic risk, energy consumption and positioning uncertainty.

[0077] Step S6: Jointly register the airborne observations with the prior of the building information model and update the pose covariance. Perform five types of multi-hypothesis tests on the abnormal registration residuals in parallel: positioning drift, model bias, new dynamic obstacles, missing or displaced components, and perceived noise. Update the pose, dynamic layer, or model bias layer according to the posterior confidence level. Classify and track dynamic obstacles and generate time slice prediction occupancy envelopes. Correct the trajectory in the forward pipeline defined by the reversible corridor certificate and verify the reverse emergency pipeline node by node.

[0078] Step S7: Update the evidence matrix and coverage debt with actual pose, effective observation data, incident angle, occlusion rate and repeatability consistency. For target patches with increased coverage debt, generate alternative viewpoints within the valid area of ​​the reversible corridor certificate and perform supplementary measurements. After all target patches have zero coverage debt and the reversible corridor certificate is valid, perform a safe exit and output component-level inspection data and map increments bound to the component's globally unique identifier.

[0079] As an optional implementation method, this embodiment takes a high-rise building scaffolding project as an example to describe in detail the complete execution process of the flying robot full-domain inspection method.

[0080] Further, step S1 includes:

[0081] Step S11: Obtain the building information model, inspection task list, robot parameters, sensor parameters, safety exit location and endurance constraints of the target project at the current construction stage, complete the model version and coordinate system verification, and confirm that there are no abnormal scales, duplicate components or missing components in the model.

[0082] To clarify, the Building Information Modeling (BIM) model is a priori map that carries the geometric and semantic information of all professional components of the high-rise building project. In this embodiment, it refers to the fact that the external scaffolding of the target project has been fully erected in the current construction stage. The inspection task list clearly requires a full-coverage inspection of the installation quality of all scaffolding members, fasteners, and wall ties, and the corresponding operation of the robot is determined based on the given robot parameters.

[0083] Step S12: Parse the geometry, category, risk level, and inspection attributes of each component in the Building Information Model, identify static obstacles, no-fly zones, and target surfaces, and establish a unified index for each component that includes geometric area, category, risk level, inspection attributes, and version information. The component semantics come from the IFC entity type, family name, component globally unique identifier, or attribute field.

[0084] Specifically, the geometry, category, risk level, construction stage, and inspection attributes of each component in the BIM are analyzed to identify the main structure, uprights, horizontal bars, scissor braces, scaffold boards, protective netting, wall ties, cantilever supports, and edge openings. Wall ties, cantilever supports, and key fasteners are set as high-weight inspection targets, and a unified index containing geometric area, category, risk level, inspection attributes, and version information is established for each component. Component semantics are derived from IFC (Industry Foundation Classes) entity type, family name, globally unique identifier (GUID), or attribute fields. For components that cannot be covered by rule-based identification, manual annotation or model identification can be used to supplement them.

[0085] Step S13: Perform multi-resolution voxelization on the building information model. Use fine-resolution voxels in areas with dense components, concentrated targets, or large clearance gradients, and use coarse-resolution voxels in open areas. Connect different resolution areas through octrees, sparse voxels, or multi-scale topology graphs.

[0086] Specifically, BIM is subjected to multi-resolution voxelization according to the complexity of the scene: fine-resolution voxels are used in densely populated areas of the frame, areas with concentrated targets, or areas with large clearance gradients, while coarse-resolution voxels are used in open areas outside the building. Different resolution areas are connected by octrees, sparse voxels, or multi-scale topology maps. Among them, fine-resolution voxels are 0.02 to 0.10 m, and coarse-resolution voxels are 0.10 to 0.50 m, which can be set according to the actual situation.

[0087] Step S14: Construct a multi-layer semantic map that includes a static occupancy layer, a dynamic prediction occupancy layer, a clearance risk layer, a topology retreat layer, and a target visibility layer. Each layer can be queried uniformly through a globally unique identifier for the component, a spatial index, and a timestamp.

[0088] Specifically, a multi-layered semantic map is constructed: First, a static occupancy layer, i.e., component semantics, is used to store the distribution of static obstacles and the semantic attributes of each component; Second, a dynamic prediction occupancy layer, which stores the predicted envelopes and existence probabilities of dynamic objects such as personnel, hoisted objects, and temporary stockpiles in future time slices; Third, a clearance risk layer, i.e., an uncertainty layer, stores the clearance of obstacle distances, semantic risk margins, and location covariance at each location; Fourth, a topology retreat layer, i.e., a reversible layer, is used to store the topology of cross-domain portal nodes, safe exit root nodes, multi-source retreat costs, and the effective interval of reversible corridor certificates; Fifth, a target visibility layer, i.e., an evidence layer, is used to store target patches, candidate viewpoints, occlusion relationships, actual observation quality, and targets with coverage liabilities. Each layer is uniformly queried through globally unique component identifiers, spatial indexes, and timestamps. Any module can directly retrieve the corresponding spatial status and evidence records by component GUID.

[0089] Further, step S2 includes:

[0090] Step S21: Obtain the collision envelope of the flying robot. The collision envelope is approximated by one of the following: circumscribed ellipsoid, circumscribed cylinder, or a combination of multiple spheres.

[0091] Specifically, the shape envelope, speed limit, and endurance parameters of the flying robot are obtained. In this embodiment, the robot collision envelope can be approximated by one of the following: circumscribed ellipsoid, circumscribed cylinder, or a combination of multiple spheres, to truly reflect the physical constraints of the robot's volume and avoid overestimating the passable clearance in the mass model. Specifically, the robot with a protective frame can be approximated as a cylinder with a radius of 0.30 m and a height of 0.25 m, with a basic safety margin of 0.15 m. The personnel passage and hoisting area are further supplemented with semantic risk margins. The relevant values ​​are examples and can be set according to the actual situation.

[0092] Step S22: Determine the obstacle expansion amount, which is jointly determined by the equivalent size of the body, the basic safety margin, the semantic risk additional margin, and the positioning uncertainty additional margin; in this step, Minkowski adaptive expansion equivalent to the robot envelope is performed on all obstacles in the BIM.

[0093] Step S23: Calculate the effective clearance at any location by subtracting the expansion amounts of the equivalent radius of the aircraft, the basic safety margin, the semantic risk margin, and the positioning uncertainty margin from the distance to the obstacle.

[0094] Specifically, the formula for calculating the effective clearance is as follows:

[0095]

[0096] in, Indicates effective clearance; Indicates the distance to the obstacle; Indicates the equivalent radius of the body; Indicates the basic safety margin; This indicates additional margin for semantic risk; This represents the uncertainty amplification factor; This represents the largest eigenvalue of the localization covariance matrix.

[0097] Step S24: Voxels with effective clearance not greater than zero are identified as inaccessible areas. The remaining voxel space with effective clearance greater than zero after expansion constitutes a safe free space. Based on this step, the safe free space is used for all path planning and viewpoint generation. It should be noted that voxels with effective clearance less than or equal to zero are identified as inaccessible areas even if there are no obstacle markings in the BIM model. Only the remaining space after expansion is used as the planning free space to avoid mistakenly simplifying a robot with volume to a point mass.

[0098] Further, step S3 includes:

[0099] Step S31: Discretize the surface of the component to be inspected into target patches according to area, curvature, risk level and detection resolution, and generate one or more candidate viewpoints for each target patch within the effective working distance of the sensor based on its surface normal.

[0100] Specifically, while determining candidate viewpoints, the camera orientation, pitch angle, and sampling parameters are also determined; multi-viewpoints with different directions are set for wall ties, cantilevered bottom surfaces, and backside nodes; high-risk targets require at least two independent and effective observations from different viewpoint clusters; optionally, the inspection operation distance can be 0.30–3.00 m, and the actual value should be adaptively determined according to sensor performance and target size. A candidate viewpoint is considered valid only if it simultaneously meets the following conditions: the viewpoint is located within the safe free space extracted in the aforementioned steps; the target distance is within the effective range of the sensor; the target falls within the sensor's field of view; there is no static or current dynamic occlusion between the target and the viewpoint; the estimated effective pixel count, point density, incident angle, or imaging quality reaches a threshold; the viewpoint is reachable from the mission starting point, and at least one safe exit is reachable from the viewpoint.

[0101] Step S32: Establish a prediction coverage matrix offline, where each element represents the expected coverage quality of the target patch by the candidate viewpoint; establish an evidence matrix online, whose elements are calculated from the actual pose, effective pixels or points, sharpness, incident angle, occlusion rate, and consistency of repeated observations.

[0102] Specifically, the target patch for the current analysis is defined as... The formula for calculating the coverage debt of the target patch is:

[0103]

[0104] in, Represents the target patch Covering debt; This indicates the operation of finding the maximum value. Represents the target patch The required number of independent valid observations; Indicates the candidate viewpoint index; Indicates characteristic functions; Represents the evidence matrix; This represents the observation quality threshold.

[0105] Step S33: Define the target patch coverage debt as the non-negative value of the difference between the number of independent valid observations required for the target patch and the actual number of observations in the evidence matrix that reach the observation quality threshold. When obstacle avoidance, attitude deviation or new occlusion causes the evidence matrix to decrease, the coverage debt will automatically increase again. High-risk patches require independent evidence from different view clusters.

[0106] Specifically, when At that time, the target surface Coverage complete; obstacle avoidance, attitude deviation, or new occlusions affect the evidence matrix. When the medium element decreases, it covers debt. Automatic re-incrementing, rather than replacing check completion with route passing; high-risk patches also require independent evidence from different view clusters; the necessary and sufficient condition for mission completion is that all critical patches meet the requirements. Furthermore, the corresponding evidence has been bound to the component GUID and timestamp.

[0107] Further, step S4 includes:

[0108] Step S41: Based on component type, clearance distribution, spatial connectivity, and target distribution, divide the task space into the internal area of ​​the frame, the external area of ​​the building, the cantilever bottom area, and the high-risk area. Set map resolution, speed limit, safety margin, and viewpoint generation strategy for each area.

[0109] Specifically, based on component type, clearance distribution, spatial connectivity, and target distribution, the mission space is divided into a narrow area 400 inside the frame, an open area 500 outside the building, a cantilevered bottom area, a no-fly or high-risk area, and a cross-domain transition area. Different areas are configured with different map resolutions, speed limits, safety margins, positioning weights, and viewpoint generation strategies: the inside of the frame uses fine resolution, low speed, and a large safety margin; the outside of the building uses coarse resolution, higher speed, and a standard margin.

[0110] Step S42: Extract cross-domain portal nodes at the region boundary and record the entry direction, attitude range, minimum clearance, nearby targets, regional risk level, and available exit direction for each portal;

[0111] Step S43: Connect the internal area of ​​the frame, the external area of ​​the building, and the cantilevered bottom area into a unified task map through the cross-domain portal, eliminating the need for manual takeover and realizing unified archiving of inspection data; In this step, the narrow area 400 inside the frame, the open area 500 outside the building, and the cantilevered bottom area are connected into a unified task map through the cross-domain portal, and are no longer treated as multiple independent routes that are not related to each other.

[0112] Further, step S5 includes:

[0113] Step S51: Calculate the spatiotemporal reverse emergency pipeline to the safe exit for each prefix node of the candidate trajectory. All states within the reverse emergency pipeline must satisfy the following conditions: the dynamic prediction occupancy probability is lower than the threshold, the effective clearance is greater than zero, and the flight control dynamics constraints are met.

[0114] Specifically, in this step, the candidate trajectory, i.e., each candidate entry trajectory, is denoted as... Each prefix node It doesn't just check the return trip at the destination point, but for every prefix node. Calculate the spatiotemporal reverse emergency pipeline It consists of a continuous set of states, and all states in the set must satisfy the condition that the dynamically predicted occupancy probability is below a threshold. Effective airspace The conditions for a positive value of zero, achievable velocity / acceleration / angular velocity, and communication or autonomous landing must be met.

[0115] Step S52: Verify three constraints: Effective clearance constraint requires that the effective clearance of all predicted states in the reverse emergency pipeline is greater than zero; Dynamic occupancy probability constraint requires that the predicted collision probability of all states in the pipeline is lower than the threshold; Energy reserve constraint requires that the sum of the exit energy consumption upper limit, return reserve energy consumption and uncertainty additional energy consumption does not exceed the current remaining energy. The exit energy consumption upper limit is estimated based on the exit path length, climb altitude, wind disturbance additional amount and hovering waiting.

[0116] Specifically, the upper limit of exit energy consumption is estimated based on exit path length, climb height, wind disturbance additional amount, and hovering waiting time; in particular, the issuance of a reversible corridor certificate must simultaneously meet the following three constraints:

[0117] Effective clearance constraint, i.e. , It represents a continuous set of states, meaning that the effective clearance of all predicted states within the reverse emergency pipeline is greater than zero.

[0118] Dynamic occupancy probability constraint, i.e. That is, the predicted collision probability of all states in the pipe is lower than the threshold.

[0119] Energy reserve constraints, i.e. , Indicates the elimination of energy consumption; This indicates the energy consumption for return flight, which is typically 15% to 35% of the rated available energy. It can be set according to the actual situation. This represents the additional energy consumption corresponding to the uncertainty of positioning and dynamic prediction. This indicates the current remaining energy.

[0120] Step S53: Issue a reversible corridor certificate to the prefix node that satisfies all constraints. The certificate includes the trajectory version, map version, effective time window, safety exit identifier, minimum effective clearance, maximum predicted collision probability, and upper limit of exit energy consumption. The certificate is immediately revoked when any field exceeds the limit or the map version changes, and the state machine switches to the execution sequence of stopping exploration, retreating to a safe node, restoring the certificate, and replanning.

[0121] Specifically, all the constraints proposed in the aforementioned steps must be satisfied before prefix nodes can be processed. Issuance of Reversible Corridor Certificate The certificate content includes: track version, map version, effective time window, safety exit sign, minimum effective clearance, maximum predicted collision probability, and upper limit of exit energy consumption; if any of the above fields exceeds the limit or the map version changes, the certificate will be invalidated. Immediate reversal occurs, and the state machine switches to the execution sequence of "stop deepening—retreat to a safe node—restore certificate—replan." Deepening is prohibited if any condition is not met, thus explicitly ensuring the robot's reversible exit capability at each critical node.

[0122] Step S54: Based on the validity of the reversible corridor certificate, construct an attributed topology graph from candidate viewpoints and cross-domain portals, and solve the global viewpoint access sequence, main safe corridor, and emergency exit tree at the cost of coverage benefits, path length, semantic risk, energy consumption, area switching cost, and positioning uncertainty; in this step, after the relevant steps are satisfied, issue the main path, safe corridor, emergency exit tree, and target list to the flying robot 300.

[0123] Further, step S6 includes:

[0124] Step S61: Jointly register the pose estimation with covariance formed by airborne vision, lidar, IMU, barometer and ranging sensor with the prior of building information model, and update the pose covariance; In this step, during the flight execution phase, the pose estimation with covariance formed by airborne vision, lidar, IMU, barometer and ranging sensor, and jointly register the plane, rod axis, node intersection and component contour with BIM, and update the pose covariance.

[0125] Step S62: Parallel maintenance of the abnormal registration residuals for the positioning drift hypothesis, model version deviation hypothesis, newly added dynamic obstacle hypothesis, design component missing or displacement hypothesis, and perception noise hypothesis. Calculate the posterior probability of each hypothesis based on the spatial consistency, duration, whether it moves with the robot, whether it is consistent with the component boundary, and multi-view repeatability of the residuals. Perform the corresponding update only when the posterior of a certain hypothesis exceeds the confidence threshold and the difference between it and the posterior of the suboptimal hypothesis reaches the confidence interval.

[0126] Specifically, when registration residuals are abnormal, the system maintains the following five types of hypotheses in parallel: H1 positioning drift hypothesis, H2 model version deviation hypothesis, H3 newly added dynamic or temporary obstacle hypothesis, H4 design component missing or displacement hypothesis, and H5 perceived noise hypothesis. The posterior probability of each hypothesis is calculated based on the spatial consistency of the residuals, duration, whether it moves with the robot, whether it is consistent with the component boundary, and multi-view repeatability. An update is performed only when the posterior probability of a hypothesis exceeds a confidence threshold and the difference between it and the posterior probability of the suboptimal hypothesis reaches a confidence interval: if H1 is true, the pose and positioning covariance are corrected; if H2 / H4 are true, the update is written to the model deviation layer but the original BIM version is retained; if H3 is true, the update is written to the dynamic prediction occupancy layer; if H5 is true, only the observation weight is reduced. When an assumption is uncertain, the depth certificate is frozen and a supplementary view is requested to avoid erroneously absorbing real component changes into the positioning transformation.

[0127] Step S63: Classify and track dynamic obstacles, generate time-slice predicted occupancy envelopes based on velocity covariance, and update the dynamic predicted occupancy layer in real time; in this step, personnel, hoisted objects, temporary stockpiles 800, swing cables and mobile equipment are classified and tracked, and time-slice predicted occupancy envelopes are generated based on velocity covariance, and the dynamic predicted occupancy layer is updated in real time.

[0128] Step S64: Correct the trajectory within the forward pipeline defined by the reversible corridor certificate, and check the reverse emergency pipeline node by node. If a dynamic obstacle blocks the exit path, the state machine immediately stops going deeper and retreats to the nearest safe node. When the certificate expires, stop going deeper and prioritize restoring reversibility.

[0129] Specifically, the local planner only corrects the trajectory within the forward pipeline defined by the current reversible corridor certificate, and simultaneously recalculates the camera orientation, evidence matrix, and reverse emergency pipelines for all affected prefix nodes: if there is still local detour space within the corridor, the local planner generates a smooth trajectory and tries to maintain the original viewpoint and camera orientation; if a critical target leaves the field of view due to detour, the corresponding target patch coverage debt is increased and it enters the waiting-for-retest queue; if dynamic obstacle 800 blocks the exit path, the state machine immediately stops its exploration, the flying robot 300 retreats to the nearest safe node, and can only continue its exploration after restoring at least one valid exit path; when the reversible corridor certificate expires, the state machine stops its exploration, retreats to a safe node, and prioritizes the restoration of reversibility, the coverage gain weight is set to zero at this stage, and the exit-priority state machine is executed immediately. Thus, dynamic obstacle avoidance is not an independent geometric detour, but is updated together with inspection evidence and reversibility.

[0130] Further, step S7 includes:

[0131] Step S71: Update the evidence matrix and coverage debt with actual pose, effective target pixels or points, occlusion ratio, incident angle, sharpness and repeat observation consistency;

[0132] Step S72: For the target patch covering the debt rebound, prioritize generating alternative viewpoints within the effective area of ​​the reversible corridor certificate and perform supplementary testing. If there is no solution locally, reselect the portal or adjust the global task order.

[0133] Step S73: If the reversible corridor certificate has expired, perform a safe exit first, retreat to a safe node or hovering area, and then perform a global replanning. Do not replace the conclusion that the check is completed with the conclusion that the path has flown.

[0134] Step S74: After all key target patches are covered, debt is cleared, and the current location reversible corridor certificate is valid, perform a safe exit or return, output component-level inspection data bound to the component's globally unique identifier, record obstacle avoidance trajectory, certificate failure reason, compensation result, and map increment, and archive.

[0135] Specifically, in the entire process, after the flying robot reaches the viewpoint, the system updates the evidence matrix and coverage debt based on the actual pose, effective pixels or points of the target, occlusion ratio, incident angle, sharpness, and consistency of repeated observations. Then, for target patches with increased coverage debt, the system prioritizes generating alternative viewpoints within the valid area of ​​the reversible corridor certificate and performs supplementary measurements. If there is no solution locally, a new portal is selected or the global task order is adjusted. Secondly, if the reversible corridor certificate has expired, a safe exit is performed first, retreating to a safe node or hovering area before performing global replanning, without replacing the conclusion that the check has been completed with the conclusion that the path has been flown. Finally, a safe exit or return to base can only be performed when the following two conditions are met: First, the coverage debt of all key target patches is cleared, and the corresponding evidence matrix elements reach the observation quality threshold; Second, the reversible corridor certificate at the current position is valid. After the mission is completed, the return path and energy reserves are re-verified, and component-level inspection data bound to the component's globally unique identifier is output. The obstacle avoidance trajectory, certificate failure reason, compensation result and map increment are recorded, and the inspection results are archived to the mission database. It should be noted that targets that cannot be safely completed are retained as incomplete, and the inspection conclusion is not replaced by the path already flown.

[0136] Understandably, firstly, upgrading BIM from a geometric background to an executable semantic state, linking component categories, risks, occupancy, clearance, visibility, and exit costs within the same multi-layered semantic map, allows planning results to be directly traced back to specific components and target surfaces, resolving the disconnect between component semantics and planning state in existing solutions. Secondly, adaptive expansion of obstacles is performed based on the robot's actual collision envelope and positioning uncertainty. Effective clearance calculation considers robot size, safety margin, semantic risk, and positioning covariance, avoiding overestimation of passable clearance by point models or coarse grids, thus improving the safety of narrow space paths. Thirdly, a coverage debt mechanism forms a closed loop between target patches, candidate viewpoints, evidence matrices, and actual observation quality, enabling the identification of missed detections caused by obstacle avoidance or occlusion and automatically generating alternative viewpoints for supplementary measurements, transforming "flying over" into "detection," solving the problem of existing solutions being unable to verify the effective coverage quality of target surfaces. Next, a reversible corridor certificate is issued for each prefix node of the candidate trajectory. The entry path and exit path are verified in pairs. At each key node, the exit cost, clearance and remaining energy are continuously compared. When the certificate expires, the exit capability is restored first, which explicitly prevents the robot from getting stuck in narrow areas.

[0137] Then, the cross-domain portal connects the interior of the framework, the exterior of the building, and the underside of the cantilever into a unified task map, enabling integrated multi-area inspection, eliminating the need for manual intervention, and archiving inspection data uniformly using globally unique component identifiers as indexes. Furthermore, the multi-hypothesis testing mechanism allows the system to utilize BIM's global structure and component identifiers while promptly reflecting new obstacles, missing components, or model deviations, taking into account both design priors and on-site changes. Finally, it reduces the necessity for personnel to enter high-altitude, narrow, and cantilevered areas, thereby reducing hazard exposure. This effect is a risk reduction, not an absolute guarantee of project safety. Each module can be distributed across onboard computing units, ground stations, or edge servers; map structures, solvers, and sensors are all replaceable, facilitating expansion to other complex spaces and multi-robot tasks with prior 3D design models, such as bridges, tunnels, factories, and storage tanks.

[0138] To better illustrate, a typical construction scenario is used, namely, a coordinated inspection between the interior of a narrow scaffold of about 1.2 m and the exterior of the building, to explain the complete operation process of steps S1 to S7 above; the task area includes the internal passage 400 of the scaffold with a typical net width of about 1.2 m, the exterior facade of the building, the cantilevered bottom side, and the wall tie area.

[0139] Specifically, in step S1, after this step is executed, the system has established a five-layer semantic map. The wall ties 700 and the cantilever supports are marked as high-risk targets, and the internal passage 400 is identified as a fine-resolution area with a large clearance gradient. In step S2, after the obstacle expansion is completed, the effective clearance of the narrowest section of the internal passage is about 0.30 m, and the system confirms that the flying robot 300 can pass through this section. In step S3, candidate viewpoints in the front and side directions are generated for the wall ties 700, requiring at least two independent and effective observations from different viewpoint clusters. In step S4, a cross-domain portal is established at the floor edge to connect the internal passage 400 with the external corridor 500 of the building into a unified task map. In step S5, a reversible corridor certificate is issued for each prefix node entering the internal passage, verifying that the sum of the exit energy consumption upper limit and the return reserve does not exceed the current remaining energy. If the clearance or energy of a node does not meet the conditions, the corresponding viewpoint is deleted or the depth is restricted, and no certificate is issued.

[0140] Next, in step S6, the aircraft execution phase, the robot observes the poles, fasteners, and wall ties 700 along the internal inspection corridor 400, binding the BIM component GUID to each valid observation. During flight, a construction worker is detected entering the passageway; the dynamic perception module models this as a dynamic expansion zone, and the local planner generates a detour trajectory within the certificate-defined forward pipeline. Because the detour causes the wall tie 700 to leave the camera's field of view, the corresponding coverage debt automatically increases, and the target re-enters the queue for supplementary testing. Since the dynamic obstacle does not block the exit path, the reversible corridor certificate remains valid, and the robot continues to penetrate deeper, supplementing the wall tie 700 after the detour, reducing the coverage debt to zero. Upon reaching the cross-domain portal, wind disturbance, positioning quality, and external clearance are reassessed, the speed limit and map resolution are switched, and the inspection of the building's outer inspection corridor 500 and the cantilevered bottom side is completed. In step S7, after verifying that all target surface areas are covered, the debt is cleared, and the certificate is valid, the robot returns along the emergency exit corridor 600, outputs a unified coverage report of the interior of the frame and the exterior of the building, and archives all inspection evidence and component GUIDs.

[0141] It should be noted that the values ​​of voxel size, safety margin, inspection operation distance, and return-to-home reserve ratio involved in the operation are only examples. The actual values ​​should be adaptively determined according to the size of the aircraft, flight control performance, wind disturbance, positioning error and sensor requirements, and do not constitute a limitation on the scope of protection of this invention.

[0142] The second embodiment of the present invention provides a full-domain inspection system for flying robots based on BIM prior semantic maps, the system comprising:

[0143] The BIM model acquisition and verification module is used to: acquire the building information model, inspection tasks, robot parameters, sensor parameters, safety exits and endurance constraints of the target project, and complete the model version and coordinate system verification.

[0144] The component semantic parsing module is used to: parse the geometry, category, risk and inspection attributes of each component, and identify static obstacles, no-fly zones and target surfaces;

[0145] The multi-layer semantic map construction module is used to: perform multi-resolution voxelization on the building information model to construct a multi-layer semantic map including a static occupancy layer, a dynamic predicted occupancy layer, a clearance risk layer, a topology avoidance layer, and a target visibility layer; obtain the collision envelope of the flying robot, determine the expansion amount based on the equivalent size of the body, the basic safety margin, the semantic risk additional margin, and the positioning uncertainty additional margin, perform adaptive expansion on obstacles, calculate the effective clearance at any position, and extract the safe free space;

[0146] The target surface and viewpoint generation module is used to: discretize the surface of the component to be inspected into target patches, generate candidate viewpoints based on the target patch normal, sensor field of view, working distance and occlusion relationship, establish a prediction coverage matrix offline, establish an evidence matrix online, use the target patch coverage debt to represent the inspection completion status, and automatically trigger supplementary testing when the coverage debt increases.

[0147] The bidirectional reachability and exit verification module is used to: calculate the spatiotemporal reverse emergency pipeline for each prefix node of the candidate trajectory, verify the effective clearance constraint, the dynamic prediction occupancy probability constraint and the exit energy consumption upper limit constraint, issue a reversible corridor certificate with map version and effective time window for nodes that meet all constraints, and immediately revoke the certificate and trigger the exit priority state machine when any constraint is exceeded.

[0148] The security corridor and global planning module is used to: solve for the global viewpoint access sequence, main security corridor, and emergency exit tree at the cost of coverage benefits, path length, semantic risks, energy consumption, and positioning uncertainty, based on the validity of the certificate;

[0149] The real-scene registration and fusion positioning module is used to: jointly register airborne observations with the prior of the building information model and update the pose covariance; perform five types of multi-hypothesis tests on abnormal registration residuals in parallel, including positioning drift, model bias, new dynamic obstacles, missing or displaced components, and perceived noise; and update the pose, dynamic layer or model bias layer according to the posterior confidence.

[0150] The dynamic semantic perception module is used to classify and track dynamic obstacles and generate time-slice predicted occupancy envelopes.

[0151] The local trajectory correction module is used to: correct the trajectory within the forward pipeline defined by the reversible corridor certificate and verify the reverse emergency pipeline node by node;

[0152] The coverage verification and data management module is used to: update the evidence matrix and coverage debt with actual pose, effective observation data, incident angle, occlusion rate and repeatability consistency; generate alternative viewpoints and perform supplementary measurements for target patches with increased coverage debt within the effective area of ​​the reversible corridor certificate; perform safe exit after all target patches have zero coverage debt and the reversible corridor certificate is valid; and output component-level inspection data and map increments bound to the component's globally unique identifier.

[0153] When the system is in operation, it needs to use a full-domain inspection method for flying robots based on BIM prior semantic maps. Therefore, whether the system and program data are integrated or different hardware is configured to produce functions with similar effects to those achieved by the present invention, they all fall within the protection scope of the present invention. The system has the same beneficial effects as the aforementioned full-domain inspection method for flying robots based on BIM prior semantic maps, which will not be elaborated here.

[0154] It should be noted that the order of the above embodiments of the present invention is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. The processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired result. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0155] The various embodiments in this specification are described in a progressive manner. The same or similar parts between the various embodiments can be referred to each other. Each embodiment focuses on describing the differences from other embodiments.

Claims

1. A method for full-domain inspection of flying robots based on BIM prior semantic maps, characterized in that, The method includes: Step S1: Obtain the building information model, inspection task, robot parameters, sensor parameters, safety exits and endurance constraints of the target project, complete the model version and coordinate system verification, analyze the geometry, category, risk and inspection attributes of each component, identify static obstacles, no-fly zones and target surfaces, perform multi-resolution voxelization on the building information model, and construct a multi-layer semantic map including a static occupancy layer, a dynamic prediction occupancy layer, a clearance risk layer, a topology avoidance layer and a target visibility layer; Step S2: Obtain the collision envelope of the flying robot, determine the expansion amount based on the equivalent size of the body, the basic safety margin, the semantic risk additional margin, and the positioning uncertainty additional margin, perform adaptive expansion on the obstacle, calculate the effective clearance at any position, and extract the safe free space. Step S3: Discretize the surface of the component to be inspected into target patches, generate candidate viewpoints based on the target patch normal, sensor field of view, working distance and occlusion relationship, establish a prediction coverage matrix offline, establish an evidence matrix online, use the coverage debt of the target patch to represent the inspection completion status, and automatically trigger supplementary testing when the coverage debt increases. Step S4: Divide the task space into the internal area of ​​the frame, the external area of ​​the building, the cantilever bottom area and the high-risk area. Extract cross-domain portals at the boundaries of the areas and record the entry direction, attitude range, minimum clearance and available exit direction. Combine the areas into a unified task map through the cross-domain portals. Step S5: Calculate the spatiotemporal reverse emergency pipeline for each prefix node of the candidate trajectory, verify the effective clearance constraint, dynamic prediction occupancy probability constraint, and exit energy consumption upper bound constraint, issue a reversible corridor certificate with map version and effective time window for nodes that satisfy all constraints, and immediately revoke the certificate and trigger the exit priority state machine when any constraint is exceeded. On the basis of valid certificate, solve the global viewpoint access sequence, main safe corridor and emergency exit tree at the cost of coverage benefit, path length, semantic risk, energy consumption and positioning uncertainty. Step S6: Jointly register the airborne observations with the prior of the building information model and update the pose covariance. Perform five types of multi-hypothesis tests on the abnormal registration residuals in parallel: positioning drift, model bias, new dynamic obstacles, missing or displaced components, and perceived noise. Update the pose, dynamic layer, or model bias layer according to the posterior confidence level. Classify and track dynamic obstacles and generate time slice prediction occupancy envelopes. Correct the trajectory in the forward pipeline defined by the reversible corridor certificate and verify the reverse emergency pipeline node by node. Step S7: Update the evidence matrix and coverage debt with actual pose, effective observation data, incident angle, occlusion rate and repeatability consistency. For target patches with increased coverage debt, generate alternative viewpoints within the valid area of ​​the reversible corridor certificate and perform supplementary measurements. After all target patches have zero coverage debt and the reversible corridor certificate is valid, perform a safe exit and output component-level inspection data and map increments bound to the component's globally unique identifier.

2. The method for full-domain inspection of flying robots based on BIM prior semantic maps according to claim 1, characterized in that, Step S1 includes: Step S11: Obtain the building information model, inspection task list, robot parameters, sensor parameters, safety exit location and endurance constraints of the target project at the current construction stage; complete the model version and coordinate system verification; and confirm that there are no abnormal scales, duplicate components or missing components in the model. Step S12: Analyze the geometry, category, risk level and inspection attributes of each component in the building information model, identify static obstacles, no-fly zones and target surfaces, and establish a unified index for each component that includes geometric area, category, risk level, inspection attributes and version information. The component semantics come from IFC entity type, family name, component globally unique identifier or attribute field. Step S13: Perform multi-resolution voxelization on the building information model. Use fine-resolution voxels in areas with dense components, concentrated targets, or large clearance gradients, and use coarse-resolution voxels in open areas. Connect areas of different resolutions through octrees, sparse voxels, or multi-scale topology graphs. Step S14: Construct a multi-layer semantic map that includes a static occupancy layer, a dynamic prediction occupancy layer, a clearance risk layer, a topology retreat layer, and a target visibility layer. Each layer can be queried uniformly through a globally unique identifier for the component, a spatial index, and a timestamp.

3. The method for full-domain inspection of flying robots based on BIM prior semantic maps according to claim 1, characterized in that, Step S2 includes: Step S21: Obtain the collision envelope of the flying robot, wherein the collision envelope is approximated by one of the following: circumscribed ellipsoid, circumscribed cylinder, or a combination of multiple spheres; Step S22: Determine the obstacle expansion amount, which is jointly determined by the equivalent size of the body, the basic safety margin, the semantic risk additional margin, and the positioning uncertainty additional margin; Step S23: Calculate the effective clearance at any location by deducting the expansion amount of the equivalent radius of the aircraft, the basic safety margin, the semantic risk margin, and the positioning uncertainty margin from the distance to the obstacle; Step S24: Voxels with effective clearance not greater than zero are identified as inaccessible areas, and the remaining voxel space with effective clearance greater than zero after expansion constitutes a safe free space.

4. The method for full-domain inspection of flying robots based on BIM prior semantic maps according to claim 1, characterized in that, Step S3 includes: Step S31: Discretize the surface of the component to be inspected into target patches according to area, curvature, risk level and detection resolution, and generate one or more candidate viewpoints for each target patch within the effective working distance of the sensor based on its surface normal. Step S32: Establish a prediction coverage matrix offline, where each element represents the expected coverage quality of the target patch by the candidate viewpoint; establish an evidence matrix online, whose elements are calculated from the actual pose, effective pixels or points, sharpness, incident angle, occlusion rate and consistency of repeated observations. Step S33: Define the target patch coverage debt as the non-negative value of the difference between the number of independent valid observations required for the target patch and the actual number of observations in the evidence matrix that reach the observation quality threshold. When obstacle avoidance, attitude deviation or new occlusion causes the evidence matrix to decrease, the coverage debt will automatically increase again. High-risk patches require independent evidence from different view clusters.

5. The method for full-domain inspection of flying robots based on BIM prior semantic maps according to claim 1, characterized in that, Step S4 includes: Step S41: Based on component type, clearance distribution, spatial connectivity, and target distribution, divide the task space into the internal area of ​​the frame, the external area of ​​the building, the cantilever bottom area, and the high-risk area. Set map resolution, speed limit, safety margin, and viewpoint generation strategy for each area. Step S42: Extract cross-domain portal nodes at the region boundary and record the entry direction, attitude range, minimum clearance, nearby targets, regional risk level, and available exit direction for each portal; Step S43: Connect the internal area of ​​the frame, the external area of ​​the building, and the cantilevered bottom area into a unified task map through a cross-domain portal, eliminating the need for manual takeover and realizing unified archiving of inspection data.

6. The method for full-domain inspection of flying robots based on BIM prior semantic maps according to claim 1, characterized in that, Step S5 includes: Step S51: Calculate the spatiotemporal reverse emergency pipeline to the safe exit for each prefix node of the candidate trajectory. All states in the reverse emergency pipeline must meet the following conditions: the dynamic prediction occupancy probability is lower than the threshold, the effective clearance is greater than zero, and the flight control dynamics constraints are met. Step S52: Verify three constraints: Effective clearance constraint requires that the effective clearance of all predicted states in the reverse emergency pipeline is greater than zero; Dynamic occupancy probability constraint requires that the predicted collision probability of all states in the pipeline is lower than the threshold; Energy reserve constraint requires that the sum of the exit energy consumption upper limit, return reserve energy consumption and uncertainty additional energy consumption does not exceed the current remaining energy. The exit energy consumption upper limit is estimated based on the exit path length, climb altitude, wind disturbance additional amount and hovering waiting time. Step S53: Issue a reversible corridor certificate to the prefix node that satisfies all constraints. The certificate includes the trajectory version, map version, effective time window, safety exit identifier, minimum effective clearance, maximum predicted collision probability, and upper limit of exit energy consumption. The certificate is immediately revoked when any field exceeds the limit or the map version changes, and the state machine switches to the execution sequence of stopping exploration, retreating to a safe node, restoring the certificate, and replanning. Step S54: Based on the validity of the reversible corridor certificate, construct an attributed topology graph from candidate viewpoints and cross-domain portals, and solve the global viewpoint access sequence, main security corridor and emergency exit tree at the cost of coverage benefits, path length, semantic risk, energy consumption, area switching cost and positioning uncertainty.

7. The method for full-domain inspection of flying robots based on BIM prior semantic maps according to claim 1, characterized in that, Step S6 includes: Step S61: Jointly register the pose estimation with covariance formed by airborne vision, lidar, IMU, barometer and ranging sensor with the prior of building information model, and update the pose covariance; Step S62: Parallel maintenance of the abnormal registration residuals for the positioning drift hypothesis, model version deviation hypothesis, newly added dynamic obstacle hypothesis, design component missing or displacement hypothesis, and perception noise hypothesis. Calculate the posterior probability of each hypothesis based on the spatial consistency, duration, whether it moves with the robot, whether it is consistent with the component boundary, and multi-view repeatability of the residuals. Only when the posterior probability of a certain hypothesis exceeds the confidence threshold and the difference between it and the posterior probability of the suboptimal hypothesis reaches the confidence interval, the corresponding update is performed. Step S63: Classify and track dynamic obstacles, generate time-slice predicted occupancy envelopes based on velocity covariance, and update the dynamic predicted occupancy layer in real time. Step S64: Correct the trajectory within the forward pipeline defined by the reversible corridor certificate, and check the reverse emergency pipeline node by node. If a dynamic obstacle blocks the exit path, the state machine immediately stops going deeper and retreats to the nearest safe node. When the certificate expires, stop going deeper and prioritize restoring reversibility.

8. The method for full-domain inspection of flying robots based on BIM prior semantic maps according to claim 1, characterized in that, Step S7 includes: Step S71: Update the evidence matrix and coverage debt with actual pose, effective target pixels or points, occlusion ratio, incident angle, sharpness and repeat observation consistency; Step S72: For the target patch covering the debt rebound, prioritize generating alternative viewpoints within the effective area of ​​the reversible corridor certificate and perform supplementary testing. If there is no solution locally, reselect the portal or adjust the global task order. Step S73: If the reversible corridor certificate has expired, perform a safe exit first, retreat to a safe node or hovering area, and then perform a global replanning. Do not replace the conclusion that the check is completed with the conclusion that the path has flown. Step S74: After all key target patches are covered, debt is cleared, and the current location reversible corridor certificate is valid, perform a safe exit or return, output component-level inspection data bound to the component's globally unique identifier, record obstacle avoidance trajectory, certificate failure reason, compensation result, and map increment, and archive.

9. A full-domain inspection system for flying robots based on BIM prior semantic maps, characterized in that, The system includes: The BIM model acquisition and verification module is used to: acquire the building information model, inspection tasks, robot parameters, sensor parameters, safety exits and endurance constraints of the target project, and complete the model version and coordinate system verification. The component semantic parsing module is used to: parse the geometry, category, risk and inspection attributes of each component, and identify static obstacles, no-fly zones and target surfaces; The multi-layer semantic map construction module is used to: perform multi-resolution voxelization on the building information model to construct a multi-layer semantic map including a static occupancy layer, a dynamic predicted occupancy layer, a clearance risk layer, a topology avoidance layer, and a target visibility layer; obtain the collision envelope of the flying robot, determine the expansion amount based on the equivalent size of the body, the basic safety margin, the semantic risk additional margin, and the positioning uncertainty additional margin, perform adaptive expansion on obstacles, calculate the effective clearance at any position, and extract the safe free space; The target surface and viewpoint generation module is used to: discretize the surface of the component to be inspected into target patches, generate candidate viewpoints based on the target patch normal, sensor field of view, working distance and occlusion relationship, establish a prediction coverage matrix offline, establish an evidence matrix online, use the target patch coverage debt to represent the inspection completion status, and automatically trigger supplementary testing when the coverage debt increases. The bidirectional reachability and exit verification module is used to: calculate the spatiotemporal reverse emergency pipeline for each prefix node of the candidate trajectory, verify the effective clearance constraint, the dynamic prediction occupancy probability constraint and the exit energy consumption upper limit constraint, issue a reversible corridor certificate with map version and effective time window for nodes that meet all constraints, and immediately revoke the certificate and trigger the exit priority state machine when any constraint is exceeded. The security corridor and global planning module is used to: solve for the global viewpoint access sequence, main security corridor, and emergency exit tree at the cost of coverage benefits, path length, semantic risks, energy consumption, and positioning uncertainty, based on the validity of the certificate; The real-scene registration and fusion positioning module is used to: jointly register airborne observations with the prior of the building information model and update the pose covariance; perform five types of multi-hypothesis tests on abnormal registration residuals in parallel, including positioning drift, model bias, new dynamic obstacles, missing or displaced components, and perceived noise; and update the pose, dynamic layer or model bias layer according to the posterior confidence. The dynamic semantic perception module is used to classify and track dynamic obstacles and generate time-slice predicted occupancy envelopes. The local trajectory correction module is used to: correct the trajectory within the forward pipeline defined by the reversible corridor certificate and verify the reverse emergency pipeline node by node; The coverage verification and data management module is used to: update the evidence matrix and coverage debt with actual pose, effective observation data, incident angle, occlusion rate and repeatability consistency; generate alternative viewpoints and perform supplementary measurements for target patches with increased coverage debt within the effective area of ​​the reversible corridor certificate; perform safe exit after all target patches have zero coverage debt and the reversible corridor certificate is valid; and output component-level inspection data and map increments bound to the component's globally unique identifier.

Images