Inspection unmanned aerial vehicle inspection control method and device and computer equipment

Abstract

The application relates to an inspection unmanned aerial vehicle inspection control method and device, computer equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: obtaining laser radar point clouds and depth camera point clouds collected by an inspection unmanned aerial vehicle for a pipe gallery, and inspection information of the inspection unmanned aerial vehicle; determining a point cloud consistency evaluation score according to the laser radar point clouds and the depth camera point clouds; determining an inspection reliability score according to the laser radar point clouds, the depth camera point clouds and the inspection information; optimizing a current inspection track according to the point cloud consistency evaluation score and the inspection reliability score to obtain an updated inspection track; and controlling the inspection unmanned aerial vehicle to inspect the pipe gallery according to the updated inspection track. The method can improve the inspection efficiency.

Description

Technical Field

[0001] This application relates to the field of autonomous control and environmental perception technology for unmanned aerial vehicles (UAVs), and in particular to an inspection control method, device, computer equipment, computer-readable storage medium, and computer program product for an inspection UAV. Background Technology

[0002] With the acceleration of urbanization, urban integrated utility tunnels, as an important part of urban infrastructure construction, accommodate various pipelines such as electricity, communication, gas, and water supply. Their safety inspection is crucial to ensuring the stable operation of the city's lifeline.

[0003] Typically, utility tunnel inspections rely mainly on manual foot patrols, which results in low inspection efficiency. Summary of the Invention

[0004] Therefore, it is necessary to provide a method, device, computer equipment, computer-readable storage medium, and computer program product for inspecting drones that can improve inspection efficiency, in order to address the aforementioned technical problems.

[0005] In a first aspect, this application provides a method for controlling an inspection drone, comprising: obtaining lidar point clouds and depth camera point clouds collected by the inspection drone on a utility tunnel, as well as inspection information of the inspection drone; determining a point cloud consistency evaluation score based on the lidar point clouds and the depth camera point clouds; determining an inspection reliability score based on the lidar point clouds, the depth camera point clouds, and the inspection information; optimizing the current inspection trajectory based on the point cloud consistency evaluation score and the inspection reliability score to obtain an updated inspection trajectory; and controlling the inspection drone to inspect the utility tunnel according to the updated inspection trajectory.

[0006] Secondly, this application also provides an inspection control device for an inspection drone, comprising: an acquisition module for acquiring lidar point clouds and depth camera point clouds collected by the inspection drone for a utility tunnel, as well as inspection information of the inspection drone; a first calculation module for determining a point cloud consistency evaluation score based on the lidar point cloud and the depth camera point cloud; a second calculation module for determining an inspection reliability score based on the lidar point cloud, the depth camera point cloud, and the inspection information; a processing module for optimizing the current inspection trajectory based on the point cloud consistency evaluation score and the inspection reliability score to obtain an updated inspection trajectory; and a control module for controlling the inspection drone to inspect the utility tunnel according to the updated inspection trajectory.

[0007] Thirdly, this application also provides a computer device, including a memory and a processor. The memory stores a computer program, and the processor executes the computer program to perform the following steps: obtaining lidar point clouds and depth camera point clouds collected by an inspection drone on a utility tunnel, as well as inspection information from the inspection drone; determining a point cloud consistency evaluation score based on the lidar point cloud and the depth camera point cloud; determining an inspection reliability score based on the lidar point cloud, the depth camera point cloud, and the inspection information; optimizing the current inspection trajectory based on the point cloud consistency evaluation score and the inspection reliability score to obtain an updated inspection trajectory; and controlling the inspection drone to inspect the utility tunnel according to the updated inspection trajectory.

[0008] Fourthly, this application also provides a computer-readable storage medium storing a computer program thereon, which, when executed by a processor, performs the following steps: obtaining lidar point clouds and depth camera point clouds collected by an inspection drone on a utility tunnel, as well as inspection information from the inspection drone; determining a point cloud consistency evaluation score based on the lidar point clouds and the depth camera point clouds; determining an inspection reliability score based on the lidar point clouds, the depth camera point clouds, and the inspection information; optimizing the current inspection trajectory based on the point cloud consistency evaluation score and the inspection reliability score to obtain an updated inspection trajectory; and controlling the inspection drone to inspect the utility tunnel according to the updated inspection trajectory.

[0009] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, performs the following steps: obtaining lidar point clouds and depth camera point clouds collected by an inspection drone on a utility tunnel, as well as inspection information from the inspection drone; determining a point cloud consistency evaluation score based on the lidar point clouds and the depth camera point clouds; determining an inspection reliability score based on the lidar point clouds, the depth camera point clouds, and the inspection information; optimizing the current inspection trajectory based on the point cloud consistency evaluation score and the inspection reliability score to obtain an updated inspection trajectory; and controlling the inspection drone to inspect the utility tunnel according to the updated inspection trajectory.

[0010] The aforementioned inspection drone control method, device, computer equipment, computer-readable storage medium, and computer program products, by acquiring lidar point clouds and depth camera point clouds collected by the inspection drone in the utility tunnel, as well as the inspection information from the drone itself, provide a diverse environmental and status data foundation for the inspection drone in utility tunnel environments denied by the Global Positioning System (GPS). This eliminates reliance on manual utility tunnel inspection, achieves automated acquisition of sensing information, and improves inspection efficiency. By determining the point cloud consistency evaluation score based on the lidar and depth camera point clouds, the spatial consistency of data collected by the two heterogeneous sensors can be quantified, ensuring high reliability of the environmental data used in subsequent inspection trajectory planning. This avoids low inspection accuracy due to errors in sensor-collected data, thereby improving inspection accuracy. By determining the inspection reliability score based on LiDAR point clouds, depth camera point clouds, and inspection information, the credibility of the overall utility tunnel environment perception and inspection positioning at the current moment can be comprehensively evaluated. This provides a decision-making basis for subsequent dynamic adjustment of the inspection trajectory, improving inspection accuracy. By optimizing the current inspection trajectory based on the point cloud consistency assessment score and the inspection reliability score, an updated inspection trajectory is obtained. Following this updated trajectory, the inspection drone is controlled to inspect the utility tunnel, achieving fully autonomous and highly robust inspection operations within the utility tunnel. This ensures the continuous and efficient execution of inspection tasks, replacing the traditional manual walking mode with automated operations, thus improving inspection efficiency. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart illustrating the inspection control method for an inspection drone in one embodiment;

[0013] Figure 2 This is a block diagram of the inspection control system in one embodiment;

[0014] Figure 3 This is a schematic diagram of the process by which the trajectory optimization planner optimizes the current inspection trajectory to obtain the updated inspection trajectory in one embodiment;

[0015] Figure 4 This is a flowchart illustrating the inspection control method for an inspection drone in another embodiment;

[0016] Figure 5 This is a structural block diagram of the inspection control device for an inspection drone in one embodiment;

[0017] Figure 6 This is an internal structural diagram of a computer device in one embodiment. Detailed Implementation

[0018] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0019] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various confidence thresholds, but these confidence thresholds are not limited by these terms. These terms are only used to distinguish between the first confidence threshold and the second confidence threshold. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the schemes, or any combination of multiple schemes.

[0020] With improvements in inspection technology, one implementation method involves generating inspection trajectories for inspection drones using a trajectory optimization planner, thereby enabling the drones to complete the inspection of utility tunnels. For example, trajectory optimization planners can include gradient-based local planners, such as EGO-Planner, which is mentioned in the paper "AnESDF-free Gradient-based Local Planner for Quadrotors". Taking EGO-Planner as an example, safe, smooth, and dynamically feasible trajectories can be generated through numerical optimization using elastic band methods and various constraints. Specifically, this planner constructs B-spline curves to represent trajectories and transforms requirements such as obstacle avoidance, dynamic feasibility, and smoothness into constraints and cost functions in numerical optimization. It then uses gradient descent to solve the problem. Key parameters in the safe distance cost function, such as the penalty threshold, weight coefficients of various cost terms, and optimization step size, are typically set during algorithm initialization and remain fixed throughout the operation. Therefore, the planner's workflow can be summarized as "perceptual mapping (Euclidean Signed Distance Field, ESDF), fixed-parameter planning, and trajectory tracking." In another implementation, a Kalman filter method combining surround-scan radar and visual ranging results can be used. By regionalizing obstacles, accurate target detection and estimation are achieved, thereby generating inspection trajectories. Yet another implementation uses multi-sensor fusion obstacle avoidance technology to generate inspection trajectories. The core of this approach is acquiring obstacle information from multiple sensors. For example, a common method is to weighted average or select the minimum value of distances measured by lidar and visual estimation to obtain a fused obstacle distance. By comparing this fusion distance with a pre-set, fixed safety threshold, if the fusion distance is less than the safety threshold, obstacle avoidance behavior (such as hovering or flying around) is triggered. Therefore, the logic is "multi-sensor data, fusion / selection, fixed threshold comparison and decision".

[0021] As shown above, by utilizing a trajectory optimization planner, high-quality inspection trajectories can be generated, balancing smoothness, dynamic feasibility, and safety. These trajectories perform well in most structured environments and have relatively high computational efficiency. Furthermore, by integrating sensors based on different physical principles, the redundancy and reliability of environmental perception are improved, making it better suited to complex environments than single-sensor solutions and enhancing inspection accuracy.

[0022] However, both the cost function weights of trajectory optimization planners (such as EGO-Planner) and the safety thresholds in fusion obstacle avoidance methods are fixed values. In utility tunnels with varying structures and lighting, fixed parameters cannot meet the needs of different scenarios. For example, in wide, straight corridors, a fixed threshold may prevent drones from getting close enough for inspection, resulting in low efficiency; in narrow bends or dense pipeline areas, the same fixed threshold may be insufficient, leading to collision risks or planning failures.

[0023] Furthermore, in the existing technical architecture, on the one hand, the perception module and the planning module are separated, and the perception quality is not considered. That is, the perception module is responsible for outputting an environmental model (such as obstacle locations), and the planning module makes decisions based on this model. However, when using a trajectory optimization planner for inspection trajectory planning, the planner has no way of knowing the reliability of the current perception model. In the utility tunnel, visual sensors will fail in dark areas, and LiDAR will produce noise on metal surfaces, all of which will lead to a decrease in the quality of the perception model. However, the existing planner will still optimize based on this "unreliable" model, which may generate an unsafe trajectory, thus reducing the accuracy of the inspection. On the other hand, the fusion strategy is simple and fails to intelligently handle sensor failures. That is, existing multi-sensor fusion mostly uses fixed weights or simple logic (such as taking the minimum value). When a sensor outputs erroneous data due to interference from the utility tunnel environment, its harmful data will still pollute the fusion result in a fixed proportion, lacking an intelligent mechanism that dynamically weights or deweights based on the real-time working status of the sensors.

[0024] As can be seen from the above, existing technologies lack overall robustness when facing the uncertain environment of utility tunnels. Once environmental disturbances cause a decline in perception quality, the decision chain of the entire inspection and control system may fail, making it impossible to achieve stable and reliable long-term autonomous inspections, thus reducing the accuracy and efficiency of inspections. Therefore, while existing inspection technologies employing trajectory optimization planners and multi-sensor fusion each have their advantages, their core problems of "fixed parameters" and "perception-planning separation" make it difficult to directly meet the extremely high requirements for adaptability and robustness in utility tunnel inspections.

[0025] In view of this, this application provides a method for controlling an inspection drone. This method can be applied to a server that is communicatively connected to an inspection control system. The inspection control system is deployed in the inspection drone and may include sensors such as LiDAR and depth cameras, as well as ultra-wideband (UWB) tagging devices. UWB tagging devices are signal transmitting or receiving devices deployed on the inspection drone for ultra-wideband positioning. The server can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server providing cloud computing services.

[0026] In one exemplary embodiment, such as Figure 1 As shown, a method for controlling the inspection of an inspection drone is provided. Taking the application of this method to a server as an example, the method includes the following steps:

[0027] S102 obtains the lidar point cloud and depth camera point cloud collected by the inspection drone for the utility tunnel, as well as the inspection information of the inspection drone.

[0028] LiDAR point cloud refers to the point cloud collected using LiDAR. For example, if the LiDAR is set on the top of an inspection drone and the scanning direction is omnidirectional, then the LiDAR point cloud specifically refers to the three-dimensional spatial point dataset generated by the inspection drone when flying in the utility tunnel, using the LiDAR on the top to perform an omnidirectional scan of the top of the utility tunnel, the pipe walls, and the internal pipes. This dataset is used to reflect the geometric contours and obstacle distribution of the utility tunnel environment.

[0029] Depth camera point cloud refers to the point cloud generated using images captured by a depth camera (such as a depth vision camera). For example, an inspection drone includes four rotors, and a depth camera is positioned at the front of the fuselage between the two forward rotors corresponding to the forward direction. Specifically, the depth camera point cloud refers to the forward 3D point cloud data generated by the depth camera at the front of the fuselage when the inspection drone is flying within a pipe gallery. This data is generated through coordinate transformation based on the acquired depth images and reflects the distribution of pipes, supports, and obstacles in front of the inspection drone.

[0030] Inspection information refers to a comprehensive data set used to describe the movement status and mission execution of an inspection drone. For example, inspection information includes the drone's position and attitude, as well as mission execution information. Mission execution information may include the progress of the current inspection mission, the amount of data collected, and detected abnormal targets (such as pipeline leaks or equipment malfunctions). This information is used to generate inspection reports or trigger specific tasks (such as hovering and taking photos).

[0031] In some embodiments, the utility tunnel is equipped with a UWB base station network, which includes multiple base stations. The inspection drone is equipped with a UWB tag device, and each base station sends a UWB signal to the UWB tag device. The inspection location of the drone within the utility tunnel can be determined by the time the UWB tag device receives each UWB signal. In other words, the UWB base station network and the UWB tag device interact via UWB signals, which carry timestamp information used to calculate the real-time location of the inspection drone within the utility tunnel.

[0032] S104. Determine the point cloud consistency evaluation score based on the LiDAR point cloud and the depth camera point cloud.

[0033] Among them, the point cloud consistency assessment score refers to the quantitative index obtained by comparing the geometric distribution or structural characteristics of the LiDAR point cloud and the depth camera point cloud, which is used to characterize the degree of matching or similarity between the two sensors (i.e., LiDAR and depth camera) in the perception results of the tunnel environment.

[0034] For example, a point cloud consistency assessment score can be obtained by calculating the overlap or distribution similarity between lidar point clouds and depth camera point clouds within overlapping spatial regions. Alternatively, a point cloud consistency assessment score can be obtained by statistically analyzing the presence of both types of point clouds within the same grid using a voxelized mesh method.

[0035] S106. Based on the lidar point cloud, depth camera point cloud, and inspection information, determine the inspection reliability score.

[0036] Among them, the inspection reliability score refers to a quantitative indicator that characterizes the overall perception and positioning reliability at the current moment by comprehensively evaluating the sensor data quality and the motion state of the inspection drone, based on the combined LiDAR point cloud, depth camera point cloud, and inspection information.

[0037] S108: Based on the point cloud consistency assessment score and the inspection reliability score, the current inspection trajectory is optimized to obtain the updated inspection trajectory.

[0038] The current inspection trajectory refers to the inspection trajectory executed by the inspection drone at the current moment.

[0039] For example, if the current time refers to the start time of the inspection, the current inspection trajectory refers to the initial inspection trajectory. The initial inspection trajectory can be obtained based on the utility tunnel map and preset inspection waypoints, or it can be obtained by processing data related to the utility tunnel using other trajectory generation methods.

[0040] For example, if the current moment refers to a certain moment in the inspection process of the inspection drone, the current inspection trajectory refers to the historical inspection trajectory obtained and output by the trajectory optimization planner at the moment before the current moment.

[0041] S110, following the updated inspection trajectory, controls the inspection drone to inspect the pipe gallery.

[0042] By employing the method described above, the inspection drone acquires LiDAR point clouds and depth camera point clouds collected by the drone, along with its own inspection information. This provides a diverse environmental and status data foundation for the drone in utility tunnel environments where GPS is denied, eliminating reliance on manual inspection and enabling automated acquisition of sensing information, thus improving inspection efficiency. By determining a point cloud consistency evaluation score based on the LiDAR and depth camera point clouds, the spatial consistency of data collected by the two heterogeneous sensors is quantified, ensuring high reliability of environmental data used in subsequent inspection trajectory planning. This avoids low inspection accuracy due to errors in sensor data, thereby improving inspection accuracy. Furthermore, by determining an inspection reliability score based on the LiDAR, depth camera, and inspection information, the reliability of overall utility tunnel environmental perception and inspection positioning at the current moment can be comprehensively assessed. This provides a decision-making basis for subsequent dynamic adjustment of the inspection trajectory, further improving inspection accuracy. By optimizing the current inspection trajectory based on the point cloud consistency assessment score and the inspection reliability score, an updated inspection trajectory is obtained. The inspection drone is then controlled to inspect the utility tunnel according to the updated inspection trajectory. This achieves fully autonomous and highly robust inspection operations of the inspection drone within the utility tunnel, ensuring that the inspection task can be executed continuously and efficiently. It realizes the replacement of the traditional manual walking mode with automated operations, which can improve inspection efficiency.

[0043] It is easy to understand that the above content is to obtain an updated inspection trajectory by optimizing the current inspection trajectory. Similarly, a new inspection trajectory can be obtained by optimizing the updated inspection trajectory. This process is repeated iteratively, so that the inspection drone can dynamically adjust its flight trajectory based on real-time perception information throughout the entire inspection process, ensuring that it can always adaptively balance inspection efficiency and flight safety in the pipe gallery environment with its changing structure and lighting.

[0044] In one embodiment, the method further includes: when the generation time between the current moment and the previous inspection trajectory reaches a preset time, obtaining the lidar point cloud and depth camera point cloud collected by the inspection drone on the pipe gallery, as well as the inspection information of the inspection drone. This enables the inspection drone to periodically update its flight path based on the latest sensing information at fixed time intervals, continuously and efficiently performing inspection tasks while ensuring flight safety.

[0045] In one embodiment, the method further includes: preprocessing the LiDAR point cloud and the depth camera point cloud to obtain preprocessed LiDAR point cloud and depth camera point cloud; using the preprocessed LiDAR point cloud as a new LiDAR point cloud and the preprocessed depth camera point cloud as a new depth camera point cloud; and returning to the step of determining a point cloud consistency evaluation score based on the LiDAR point cloud and the depth camera point cloud. Therefore, by preprocessing the LiDAR point cloud and the depth camera point cloud, the accuracy of trajectory planning can be improved, further enhancing the accuracy of inspection.

[0046] Preprocessing may include, but is not limited to: timestamp alignment, coordinate system unification, and filtering and denoising.

[0047] In one embodiment, the current inspection trajectory is optimized based on the point cloud consistency assessment score and the inspection reliability score to obtain an updated inspection trajectory, including the following steps:

[0048] S11, the point cloud consistency assessment score and the inspection reliability score are weighted and summed to obtain the fusion confidence score.

[0049] For example, the fusion confidence level satisfies:

[0050]

[0051] in, This represents the point cloud consistency assessment score. The inspection reliability score is represented by k. k represents the weighting coefficient used to balance the point cloud consistency assessment score and the inspection reliability score. For example, in the relevant embodiments of this application, k can be 0.6 or other values. This represents the fusion confidence level, which is in the range [0, 1]. The higher the fusion confidence level, the more reliable the entire inspection and control system's description of the utility tunnel environment at the current moment.

[0052] S12, determine the dynamic safety distance threshold, gradient optimization step size, and dynamic weights that match the fusion confidence.

[0053] S13. With the dynamic safety distance threshold, gradient optimization step size and dynamic weights input into the cost function of the trajectory optimization planner, the current inspection trajectory is optimized to obtain the updated inspection trajectory output by the trajectory optimization planner.

[0054] The cost function is a multivariate function that includes a dynamic safety distance threshold, a gradient optimization step size, and dynamic weights. Thus, by inputting the dynamic safety distance threshold, gradient optimization step size, and dynamic weights that match the fusion confidence into the cost function, the current inspection trajectory can be dynamically adjusted to obtain an updated inspection trajectory.

[0055] It should be noted that this application has made key modifications to the standard EGO-Planner, such as adding a global guiding term (i.e., dynamic weight) to the cost function of the EGO-Planner. This allows the inspection trajectory to be attracted to the center of the "global safety corridor" defined by the UWB positioning and pre-stored utility tunnel model, thereby improving inspection safety.

[0056] The method described in the above embodiments obtains a fusion confidence score by weighted summation of the point cloud consistency assessment score and the inspection reliability score. This achieves quantitative fusion and dynamic evaluation of the credibility of multi-source sensing data, transforming previously fragmented sensing quality information into a unified confidence index, providing a decision-making basis for subsequent adaptive parameter adjustments. By determining a dynamic safety distance threshold, gradient optimization step size, and dynamic weights that match the fusion confidence score, the limitation of fixed parameters in traditional methods is overcome. This allows the core parameters of the planner to change in real time with the fusion confidence score, improving inspection accuracy. By inputting the dynamic safety distance threshold, gradient optimization step size, and dynamic weights into the cost function of the trajectory optimization planner, the current inspection trajectory is optimized, resulting in an updated inspection trajectory output by the trajectory optimization planner. Thus, the fusion confidence score is directly injected into the inspection trajectory planning decision process through parameter adjustment, solving the problem of the separation between the sensing module and the planning module. This allows the optimized inspection trajectory to adapt to changes in illumination, structural differences, and sensor interference within the utility tunnel, improving inspection accuracy.

[0057] In one embodiment, by inputting a dynamic safety distance threshold, gradient optimization step size, and dynamic weights into the cost function of the trajectory optimization planner, the current inspection trajectory is optimized to obtain the updated inspection trajectory output by the trajectory optimization planner (i.e., S13), including the following steps:

[0058] S131, obtain the fused point cloud corresponding to the lidar point cloud and the depth camera point cloud.

[0059] For example, the lidar point cloud and the depth camera point cloud are fused to obtain the fused point cloud corresponding to the lidar point cloud and the depth camera point cloud.

[0060] S132, Spatial construction is performed on the inspection 3D map corresponding to the fused point cloud to obtain the inspection spatial map.

[0061] For example, a map is constructed from the fused point cloud to obtain a corresponding inspection 3D map. Further, a spatial model is constructed from the inspection 3D map to obtain an inspection spatial map. The inspection spatial map can refer to an Euclidean Signed Distance Field (ESDF) map.

[0062] S133: With the dynamic safety distance threshold, gradient optimization step size and dynamic weights input into the cost function of the trajectory optimization planner, the current inspection trajectory is optimized based on the tunnel structure data and inspection space map to obtain the updated inspection trajectory output by the trajectory optimization planner.

[0063] Among them, the structural data of the utility tunnel refers to the set of basic data describing the geometry, spatial layout, distribution of internal components, and physical boundaries of the utility tunnel.

[0064] In some embodiments, when the dynamic safety distance threshold, gradient optimization step size, and dynamic weights are input into the cost function of the trajectory optimization planner, the current inspection trajectory is optimized based on the pipe gallery structure data and the inspection space map to obtain the updated inspection trajectory output by the trajectory optimization planner. This includes: when the dynamic safety distance threshold, gradient optimization step size, and dynamic weights are input into the cost function of the trajectory optimization planner, the current inspection trajectory is optimized based on the pipe gallery structure data and the inspection space map using the gradient optimization solver in the trajectory optimization planner to obtain the updated inspection trajectory output by the trajectory optimization planner.

[0065] By employing the method described in the above embodiments, a fused point cloud corresponding to the LiDAR point cloud and the depth camera point cloud is obtained. This allows for spatiotemporal alignment and fusion of the perception data from different sensors, fully leveraging the geometric accuracy advantage of LiDAR and the texture information advantage of the depth camera. This provides a more complete and reliable data foundation for subsequent mapping, fundamentally improving the accuracy and robustness of the inspection drone. Furthermore, by constructing a spatial inspection 3D map corresponding to the fused point cloud, an inspection space map is obtained. This transforms the real-time fused point cloud into a structured environmental representation, providing accurate perception input for subsequent trajectory optimization. By inputting a dynamic safety distance threshold, gradient optimization step size, and dynamic weights into the cost function of the trajectory optimization planner, the current inspection trajectory is optimized based on the pipe gallery structure data and the inspection space map, resulting in an updated inspection trajectory output by the trajectory optimization planner. This deeply integrates static pipe gallery structure data with dynamic real-time perception maps, enabling the planner to make comprehensive decisions using both known environmental information and current perception results. This design breaks the limitations of traditional planners that rely on fixed parameters and that separate perception from planning, allowing inspection drones to adaptively adjust their inspection behavior in complex and ever-changing pipe gallery environments, thus improving inspection accuracy.

[0066] In one embodiment, determining the dynamic safety distance threshold, gradient optimization step size, and dynamic weights (i.e., S12) that match the fusion confidence includes the following steps:

[0067] S121, when the fusion confidence is greater than the first confidence threshold, the preset first safe distance threshold, the first gradient optimization step size and the first dynamic weight are respectively determined as the dynamic safe distance threshold, the gradient optimization step size and the dynamic weight that match the fusion confidence.

[0068] It is understandable that when the fusion confidence is greater than the first confidence threshold, it indicates that the perception of the utility tunnel environment is clear and reliable. Therefore, an efficiency-first inspection strategy can be adopted, that is, the preset first safety distance threshold, first gradient optimization step size and first dynamic weight are determined as the dynamic safety distance threshold, gradient optimization step size and dynamic weight that match the fusion confidence, respectively, so as to use the planner to generate the corresponding updated inspection trajectory.

[0069] S122, when the fusion confidence is less than or equal to the first confidence threshold and greater than the second confidence threshold, the preset second safety distance threshold, second gradient optimization step size and second dynamic weight are respectively determined as the dynamic safety distance threshold, gradient optimization step size and dynamic weight that match the fusion confidence; the second safety distance threshold is greater than the first safety distance threshold, the second gradient optimization step size is less than the first gradient optimization step size and the second dynamic weight is greater than the first dynamic weight.

[0070] It is understandable that when the fusion confidence is less than or equal to the first confidence threshold and greater than the second confidence threshold, it indicates that there is a certain degree of uncertainty in the perception of the utility tunnel environment. Therefore, a balanced inspection strategy can be adopted, that is, the preset second safety distance threshold, second gradient optimization step size and second dynamic weight are respectively determined as the dynamic safety distance threshold, gradient optimization step size and dynamic weight that match the fusion confidence, so as to use the planner to generate the corresponding updated inspection trajectory.

[0071] S123, when the fusion confidence is less than or equal to the second confidence threshold, the preset third safety distance threshold, third gradient optimization step size and third dynamic weight are respectively determined as the dynamic safety distance threshold, gradient optimization step size and dynamic weight that match the fusion confidence; the third safety distance threshold is greater than the second safety distance threshold, the third gradient optimization step size is less than the second gradient optimization step size, and the third dynamic weight is greater than the second dynamic weight.

[0072] For example, the third safety distance threshold can be the maximum preset safety distance threshold.

[0073] It is understandable that when the fusion confidence is less than or equal to the second confidence threshold, it indicates that the environmental perception quality of the utility tunnel is poor, the environment is complex, or there is serious interference. Therefore, a safety-first inspection strategy can be adopted, that is, the preset third safety distance threshold, third gradient optimization step size, and third dynamic weight are determined as the dynamic safety distance threshold, gradient optimization step size, and dynamic weight that match the fusion confidence, respectively, so as to use the planner to generate the corresponding updated inspection trajectory.

[0074] In one embodiment, the method further includes: triggering an active deceleration strategy, a hovering strategy, or a strategy to send an alarm to the ground station when the fusion confidence level is less than or equal to a second confidence threshold. This proactively reduces flight risk when perception quality deteriorates, ensuring the safety of the utility tunnel inspection mission.

[0075] By using the method described in the above embodiments, the core parameters of the planner can change in real time with the fusion confidence level by comparing the fusion confidence level with the first confidence level and the second confidence level, thereby improving the accuracy of the inspection.

[0076] In one embodiment, determining the dynamic safety distance threshold, gradient optimization step size, and dynamic weights (i.e., S12) that match the fusion confidence includes the following steps:

[0077] S124, Based on the fusion confidence, a dynamic safety distance threshold matching the fusion confidence is obtained through the first parameter relationship between the preset fusion confidence and the dynamic safety distance threshold.

[0078] The first parameter relationship can refer to a mapping relationship or a function relationship.

[0079] For example, the first parameter relationship satisfies:

[0080]

[0081] in, Indicates the dynamic safety distance threshold. Indicates the fusion confidence level. and These are the preset minimum and maximum safe distances. It is an adjustment factor greater than 1, used to adjust the shape of the trajectory curve.

[0082] It is easy to understand that when When it approaches 1 (i.e., the perception is highly reliable), Approaching This allows the planner to generate trajectories that closely follow obstacles, maximizing inspection efficiency. When When it approaches 0 (i.e., perception is unreliable), Approaching The planner generates a conservative trajectory that avoids all obstacles to ensure inspection safety.

[0083] S125, Based on the fusion confidence, the gradient optimization step size matching the fusion confidence is obtained through the second parameter relationship between the preset fusion confidence and the gradient optimization step size.

[0084] The second parameter can refer to a mapping relationship or a functional relationship.

[0085] For example, the relationship of the second parameter satisfies:

[0086]

[0087] in, Indicates the gradient optimization step size. Indicates the basic step size. represents the confidence threshold, and k represents the weighting coefficient.

[0088] It's easy to understand that since the trajectory optimization planner generates trajectories using gradient descent, the gradient optimization step size determines the optimization speed and stability of the trajectory. For example, when... A larger step size indicates that the gradient direction of the ambient distance field (ESDF) is reliable, which can be achieved by increasing the step size. This can accelerate trajectory optimization convergence. When When the distance is small, the ambient distance field may be noisy; this can be addressed by reducing the step size. Avoiding large steps in the wrong gradient direction that could lead to trajectory collisions can improve the robustness of trajectory optimization.

[0089] S126. Based on the fusion confidence, the dynamic weights that match the fusion confidence are obtained through the third parameter relationship between the preset fusion confidence and the dynamic weights.

[0090] The third parameter can refer to a mapping relationship or a functional relationship.

[0091] For example, the third parameter relationship satisfies:

[0092]

[0093] in, Indicates dynamic weights. This represents the basic weight.

[0094] Understandably, when local perception is unreliable (i.e.) When the value is low, increase the dynamic weight. This allows inspection drones to place greater trust in and reliance on known global prior information (such as "flying along the corridor center"). When local perception is clear and reliable (i.e. When the value is high, reduce the dynamic weight. This allows the planner to perform more precise obstacle avoidance in localized areas.

[0095] By using the method described in the above embodiments, the efficiency of obtaining a dynamic safety distance threshold, gradient optimization step size, and dynamic weight that match the fusion confidence level can be improved by pre-setting preset parameter relationships, thereby further improving inspection efficiency.

[0096] In one embodiment, the inspection information includes the inspection location, and the utility tunnel is equipped with an ultra-wideband base station network, which includes multiple base stations. Based on the LiDAR point cloud, depth camera point cloud, and inspection information, the inspection reliability score is determined (i.e., S106), including the following steps:

[0097] S1061, analyze the image corresponding to the point cloud of the depth camera, and determine the image illumination score and texture richness score of the image.

[0098] In some embodiments, analyzing the image corresponding to the depth camera point cloud to determine the image illuminance score includes: performing grayscale conversion on the image corresponding to the depth camera point cloud to obtain a grayscale image; calculating the average grayscale value of all pixels in the grayscale image as the overall illuminance value; if the overall illuminance value is less than the minimum acceptable illuminance threshold, it is determined to be too dark, and the image illuminance score is set to 0; if the overall illuminance value is greater than or equal to the minimum acceptable illuminance threshold, but less than the ideal illuminance lower limit threshold, then... The image illuminance score is positively correlated with the overall illuminance value. If the overall illuminance value is greater than or equal to the lower limit of the ideal illuminance threshold and less than or equal to the upper limit of the ideal illuminance threshold, the image is considered moderately lit, and the image illuminance score is set to 1. If the overall illuminance value is greater than the upper limit of the ideal illuminance threshold and less than or equal to the highest acceptable illuminance threshold, the image illuminance score is negatively correlated with the overall illuminance value. If the overall illuminance value is greater than the highest acceptable illuminance threshold, the image is considered overexposed, and the image illuminance score is set to 0.

[0099] For example, image illuminance scoring satisfy:

[0100]

[0101] in, Indicates the overall illuminance value. This indicates the minimum acceptable light intensity threshold. This represents the lower limit threshold of ideal illuminance. This represents the upper limit threshold of ideal illuminance. This indicates the highest acceptable light intensity threshold.

[0102] In some embodiments, analyzing the image corresponding to the depth camera point cloud to determine the texture richness score of the image includes: performing grayscale conversion on the image corresponding to the depth camera point cloud to obtain a grayscale image; performing gradient operation on the grayscale image to obtain a gradient image; calculating the gradient statistics of the gradient image, wherein the gradient statistics include at least one of gradient mean, gradient variance, or gradient energy; and determining the texture richness score based on the gradient statistics, wherein the texture richness score is positively correlated with the gradient statistics.

[0103] For example, texture richness score satisfy:

[0104]

[0105] Where 'a' represents a preset scale parameter. This represents the variance of the gradient image.

[0106] S1062, determine the visual effectiveness score of the matching between the image illuminance score and the texture richness score.

[0107] For example, a weighted sum of the image illuminance score and texture richness score is performed to obtain a visual effectiveness score that matches the image illuminance score and texture richness score.

[0108] S1063, Analyze the lidar point cloud to determine the average density of the lidar point cloud and the proportion of reflectivity anomalies in the lidar point cloud.

[0109] In some embodiments, analyzing the lidar point cloud to determine the average density of the lidar point cloud includes: dividing the space where the lidar point cloud is located into multiple voxel grids; counting the number of voxel grids occupied by at least one point and the total number of points in the lidar point cloud; and determining the ratio of the total number of points to the number of voxel grids as the average density of the lidar point cloud.

[0110] In some embodiments, analyzing the lidar point cloud to determine the proportion of reflectivity anomalies in the lidar point cloud includes: for each point in the lidar point cloud, obtaining the reflectivity value of each point in the lidar point cloud; identifying points with reflectivity values ​​lower than a preset threshold as reflectivity anomalies; counting the number of reflectivity anomalies; and determining the proportion of reflectivity anomalies in the lidar point cloud as the ratio of the number of reflectivity anomalies to the total number of points in the lidar point cloud.

[0111] The reflectivity value of each point in the lidar point cloud refers to the quantized value corresponding to the intensity of the echo signal reflected back to the receiver after the laser beam emitted by the lidar illuminates the surface of the object. It is used to characterize the reflectivity of the object's surface to laser light.

[0112] S1064, determine the effectiveness score of the lidar that matches the average density and ratio.

[0113] In some embodiments, determining a lidar effectiveness score that matches the average density and ratio includes: determining a density score by dividing a first ratio between the average density and a preset reference density value by the minimum value of 1; determining a second ratio between the ratio and a preset reference anomaly ratio value; determining an anomaly score by dividing the difference between 1 and the second ratio by the maximum value of 0; and determining a lidar effectiveness score that matches the average density and ratio by multiplying the density score and the anomaly score.

[0114] S1065 determines the UWB effectiveness score based on the inspection location and the signal-to-noise ratio of each base station.

[0115] In some embodiments, determining the ultra-wideband (UWB) effectiveness score based on the inspection location and the signal-to-noise ratio (SNR) of each base station includes: determining the distance between the inspection drone and each base station based on the inspection location; initially identifying base stations with an SNR greater than a set threshold as candidate base stations; identifying base stations with a distance less than or equal to a preset distance from the candidate base stations as valid base stations; counting the number of valid base stations; setting the UWB effectiveness score to a first preset value when the number of base stations is greater than or equal to a preset number of base stations; and setting the UWB effectiveness score to be positively correlated with the number of base stations when the number of base stations is less than the preset number of base stations.

[0116] For example, when the number of base stations is less than the preset number of base stations, a third ratio between the number of base stations and the preset number of base stations is determined; the minimum value between the third ratio and 1 is determined as the ultra-wideband effectiveness score.

[0117] S1066 calculates the inspection reliability score by weighting and summing the visual effectiveness score, lidar effectiveness score, and ultra-wideband effectiveness score.

[0118] By using the method described above, the inspection reliability score is obtained through comprehensive analysis from different dimensions. This can provide a basis for dynamic parameter adjustment of the subsequent trajectory optimization planner, which helps to improve the accuracy of inspection control.

[0119] As shown above, this application proposes a closed-loop perception-planning architecture based on "fusion confidence" feedback. Its core idea is to quantitatively evaluate the real-time output quality of multi-sensor data using a carefully designed "fusion confidence (C_fuse)" metric, and to use this evaluation result as core feedback to dynamically adjust the key parameters of the backend EGO-Planner trajectory planner, thereby achieving adaptive closed-loop control from perception to planning.

[0120] In conjunction with the above, in one embodiment, such as Figure 2 The diagram illustrates a system architecture for an inspection and control system. The system is deployed on an inspection drone and includes an onboard sensor module and a dynamic fusion and confidence calculation module. The onboard sensor module includes UWB tag devices, a depth camera, and a LiDAR. The dynamic fusion and confidence calculation module outputs an evaluation result that reflects not only "what is there" but also "how reliable this information is." Specifically, the module includes a data synchronization and preprocessing module, a consistency and reliability assessment module, and a fusion confidence calculation module. The inspection and control system communicates with an environment and infrastructure module. This module is configured with a utility tunnel structure model and a UWB base station network. The UWB base station network includes multiple base stations and communicates with the UWB tag devices via UWB signals. Therefore, based on the utility tunnel structure model, the utility tunnel structure data can be obtained.

[0121] In the airborne sensor module, the inspection control system sends signals to the UWB tag device via the UWB base station network to determine the inspection location of the inspection drone. A depth camera is used by the inspection drone to collect depth camera point clouds of the pipe gallery, and a lidar is used by the inspection drone to collect lidar point clouds of the pipe gallery.

[0122] In the dynamic fusion and confidence calculation module, the data synchronization and preprocessing module synchronizes point clouds from the airborne sensor module in real time and preprocesses the synchronized LiDAR and depth camera point clouds to improve the accuracy of trajectory planning. The consistency and reliability assessment module determines the point cloud consistency assessment score based on the LiDAR and depth camera point clouds (e.g., preprocessed LiDAR and depth camera point clouds) and determines the inspection reliability score based on the LiDAR point clouds, depth camera point clouds, and inspection information. The fusion confidence calculation module performs a weighted sum of the point cloud consistency assessment score and the inspection reliability score to obtain the fusion confidence score.

[0123] Furthermore, a dynamic safety distance threshold, gradient optimization step size, and dynamic weights matching the fusion confidence are determined; a fused point cloud corresponding to the LiDAR point cloud and depth camera point cloud is obtained, and a spatial construction is performed on the inspection 3D map corresponding to the fused point cloud to obtain an inspection spatial map (such as an ESDF map). Then, with the dynamic safety distance threshold, gradient optimization step size, and dynamic weights input into the cost function of the trajectory optimization planner, the current inspection trajectory is optimized using the gradient optimization solver in the trajectory optimization planner based on the pipe gallery structure data and the inspection spatial map, resulting in an updated inspection trajectory output by the trajectory optimization planner.

[0124] In one embodiment, the trajectory optimization planner optimizes the current inspection trajectory to obtain the updated inspection trajectory, which can be achieved as follows: Figure 3 As shown, where:

[0125] Given a defined fusion confidence level, based on the first parameter relationship between the preset fusion confidence level and the dynamic safety distance threshold, the second parameter relationship between the preset fusion confidence level and the gradient optimization step size, and the third parameter relationship between the preset fusion confidence level and the dynamic weights, a dynamic safety distance threshold, a gradient optimization step size, and dynamic weights matching the fusion confidence level can be obtained. Furthermore, by inputting the dynamic safety distance threshold, gradient optimization step size, and dynamic weights into the cost function of the trajectory optimization planner, the current inspection trajectory is optimized based on the tunnel structure data and the inspection space map (such as an ESDF map), resulting in an updated inspection trajectory output by the trajectory optimization planner.

[0126] The specific details of the first parameter relationship, the second parameter relationship, and the third parameter relationship can be found in the aforementioned description and will not be repeated here.

[0127] In one embodiment, such as Figure 4 As shown, a flowchart illustrating a method for controlling an inspection drone is provided, wherein:

[0128] S402 acquires lidar point clouds and depth camera point clouds collected by the inspection drone for the utility tunnel, as well as inspection information from the inspection drone.

[0129] S404, determine the point cloud consistency assessment score based on the LiDAR point cloud and the depth camera point cloud.

[0130] S406 determines the inspection reliability score based on LiDAR point cloud, depth camera point cloud, and inspection information.

[0131] S404 and S406 can be performed simultaneously to improve analysis efficiency, which in turn can further improve inspection efficiency.

[0132] S408 calculates the fusion confidence score by weighting and summing the point cloud consistency assessment score and the inspection reliability score.

[0133] S410, determine the dynamic safety distance threshold, gradient optimization step size, and dynamic weights that match the fusion confidence.

[0134] In one embodiment, determining the dynamic safety distance threshold, gradient optimization step size, and dynamic weights matching the fusion confidence includes: when the fusion confidence is greater than a first confidence threshold, determining a preset first safety distance threshold, a first gradient optimization step size, and a first dynamic weight as the dynamic safety distance threshold, gradient optimization step size, and dynamic weights matching the fusion confidence, respectively. When the fusion confidence is less than or equal to the first confidence threshold and greater than a second confidence threshold, determining a preset second safety distance threshold, a second gradient optimization step size, and a second dynamic weight as the dynamic safety distance threshold, gradient optimization step size, and dynamic weights matching the fusion confidence, respectively. When the fusion confidence is less than or equal to the second confidence threshold, determining a preset third safety distance threshold, a third gradient optimization step size, and a third dynamic weight as the dynamic safety distance threshold, gradient optimization step size, and dynamic weights matching the fusion confidence, respectively.

[0135] Specifically, the second safety distance threshold is greater than the first safety distance threshold, the second gradient optimization step size is less than the first gradient optimization step size, and the second dynamic weight is greater than the first dynamic weight; the third safety distance threshold is greater than the second safety distance threshold, the third gradient optimization step size is less than the second gradient optimization step size, and the third dynamic weight is greater than the second dynamic weight.

[0136] In one embodiment, determining the dynamic safety distance threshold, gradient optimization step size, and dynamic weights that match the fusion confidence includes: inputting the fusion confidence into a first parameter relationship between a preset fusion confidence and a dynamic safety distance threshold to obtain the dynamic safety distance threshold that matches the fusion confidence; inputting the fusion confidence into a second parameter relationship between a preset fusion confidence and a gradient optimization step size to obtain the gradient optimization step size that matches the fusion confidence; and inputting the fusion confidence into a third parameter relationship between a preset fusion confidence and dynamic weights to obtain the dynamic weights that match the fusion confidence.

[0137] S412 obtains the fused point cloud corresponding to the LiDAR point cloud and the depth camera point cloud.

[0138] S414: Spatial construction is performed on the inspection 3D map corresponding to the fused point cloud to obtain the inspection spatial map.

[0139] S416, with the dynamic safety distance threshold, gradient optimization step size and dynamic weight input into the cost function of the trajectory optimization planner, the current inspection trajectory is optimized according to the pipe gallery structure data and inspection space map to obtain the updated inspection trajectory output by the trajectory optimization planner.

[0140] S418, following the updated inspection trajectory, controls the inspection drone to inspect the pipe gallery.

[0141] The specific content of S402-S418 can be found in the aforementioned description and will not be repeated here.

[0142] In summary, this application provides an obstacle avoidance system for unmanned aerial vehicles (UAVs) in complex indoor environments, particularly in urban utility tunnels, where GPS denial is a concern. Furthermore, this application provides a multi-sensor data fusion method based on LiDAR, depth vision cameras, and ultra-wideband (UWB) wireless positioning technology, combined with an improved gradient-optimized trajectory planner with dynamically adjustable parameters, thereby achieving adaptive obstacle avoidance control for safe and efficient flight of UAVs in complex and uncertain environments. In other words, the method provided in this application can solve the core technical problems of poor adaptability, low intelligence level, and difficulty in balancing safety and efficiency in obstacle avoidance systems for inspection UAVs in utility tunnel inspections due to environmental complexity, variability, and uncertainty. Specifically, this application aims to address: how to enable inspection drones to assess the reliability of their own "perception capabilities" in real time, rather than just the perceived distance to obstacles; how to deeply couple the aforementioned perception reliability with the decision logic of the trajectory planner, enabling the planner to dynamically adjust the "conservatism" of its behavior based on environmental "visibility"; and how to design a mechanism that adaptively switches between safety and efficiency, allowing the drone to boldly approach and fly in favorable environments to improve inspection efficiency and accuracy, while automatically flying conservatively in adverse environments to ensure safety. Therefore, this application introduces fusion confidence (C_fuse) as a core feedback variable, deeply coupling it with a trajectory optimization planner (such as the EGO-Planner planner), achieving a fundamental shift from an open-loop "perception-planning" chain to an intelligent closed-loop "assessment-adjustment." Specifically, this application can dynamically and smoothly adjust the safety distance threshold, optimize the step size, and the global guidance weight (i.e., dynamic weight) based on real-time perception quality (C_fuse), thereby enabling the inspection drone to exhibit excellent adaptability in utility tunnel inspections. For example, in a clear, straight corridor, the system automatically adopts a small safety distance, allowing the inspection drone to fly close to the pipe wall to obtain the best inspection perspective, significantly improving inspection efficiency and quality. In areas with dim lighting, complex structures, or sensor interference, the system automatically switches to a conservative mode, expanding the safety boundary, optimizing the step size, and increasing the reliance on the global prior path. Thus, while ensuring absolute flight safety, it significantly improves the planning success rate and overall robustness of the system in extreme scenarios such as narrow bends and dense pipelines, effectively solving the core problem that traditional fixed-parameter algorithms cannot balance safety, efficiency, and adaptability in complex environments.

[0143] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0144] Based on the same inventive concept, this application also provides an inspection drone inspection control device for implementing the inspection drone inspection control method described above. The solution provided by this device is similar to the implementation scheme described in the above method; therefore, the specific limitations of one or more inspection drone inspection control device embodiments provided below can be found in the limitations of the inspection drone inspection control method described above, and will not be repeated here.

[0145] In one exemplary embodiment, such as Figure 5 As shown, an inspection control device for an inspection drone is provided, comprising: an acquisition module 502, a first calculation module 504, a second calculation module 506, a processing module 508, and a control module 510, wherein: the acquisition module 502 is used to acquire lidar point clouds and depth camera point clouds collected by the inspection drone for the pipe gallery, as well as the inspection information of the inspection drone; the first calculation module 504 is used to determine a point cloud consistency evaluation score based on the lidar point cloud and the depth camera point cloud; the second calculation module 506 is used to determine an inspection reliability score based on the lidar point cloud, the depth camera point cloud, and the inspection information; the processing module 508 is used to optimize the current inspection trajectory based on the point cloud consistency evaluation score and the inspection reliability score to obtain an updated inspection trajectory; and the control module 510 is used to control the inspection drone to inspect the pipe gallery according to the updated inspection trajectory.

[0146] In one embodiment, the processing module 508 is further configured to: perform a weighted summation of the point cloud consistency assessment score and the inspection reliability score to obtain a fusion confidence score; determine a dynamic safety distance threshold, gradient optimization step size, and dynamic weight that match the fusion confidence score; and optimize the current inspection trajectory by inputting the dynamic safety distance threshold, the gradient optimization step size, and the dynamic weight into the cost function of the trajectory optimization planner to obtain the updated inspection trajectory output by the trajectory optimization planner.

[0147] In one embodiment, the processing module 508 is further configured to: obtain a fused point cloud corresponding to the lidar point cloud and the depth camera point cloud; construct a spatial inspection 3D map corresponding to the fused point cloud to obtain an inspection spatial map; and optimize the current inspection trajectory based on the pipe gallery structure data and the inspection spatial map, while inputting the dynamic safety distance threshold, the gradient optimization step size, and the dynamic weight into the cost function of the trajectory optimization planner, to obtain the updated inspection trajectory output by the trajectory optimization planner.

[0148] In one embodiment, the processing module 508 is further configured to: when the fusion confidence is greater than a first confidence threshold, determine a preset first safety distance threshold, a first gradient optimization step size, and a first dynamic weight as a dynamic safety distance threshold, a gradient optimization step size, and a dynamic weight that match the fusion confidence, respectively; when the fusion confidence is less than or equal to the first confidence threshold and greater than a second confidence threshold, determine a preset second safety distance threshold, a second gradient optimization step size, and a second dynamic weight as a dynamic safety distance threshold, a gradient optimization step size, and a dynamic weight that match the fusion confidence, respectively; the second The safe distance threshold is greater than the first safe distance threshold, the second gradient optimization step size is less than the first gradient optimization step size, and the second dynamic weight is greater than the first dynamic weight; when the fusion confidence is less than or equal to the second confidence threshold, the preset third safe distance threshold, third gradient optimization step size, and third dynamic weight are respectively determined as the dynamic safe distance threshold, gradient optimization step size, and dynamic weight that match the fusion confidence; the third safe distance threshold is greater than the second safe distance threshold, the third gradient optimization step size is less than the second gradient optimization step size, and the third dynamic weight is greater than the second dynamic weight.

[0149] In one embodiment, the processing module 508 is further configured to: obtain a dynamic safety distance threshold matching the fusion confidence level based on the fusion confidence level using a first parameter relationship between a preset fusion confidence level and a dynamic safety distance threshold; obtain a gradient optimization step size matching the fusion confidence level based on the fusion confidence level using a second parameter relationship between a preset fusion confidence level and a gradient optimization step size; and obtain dynamic weights matching the fusion confidence level based on the fusion confidence level using a third parameter relationship between a preset fusion confidence level and dynamic weights.

[0150] In one embodiment, the inspection information includes the inspection location, the utility tunnel is deployed with an ultra-wideband base station network, and the ultra-wideband base station network includes multiple base stations; the second calculation module 506 is further configured to: analyze the image corresponding to the depth camera point cloud to determine the image illuminance score and texture richness score of the image; determine a visual effectiveness score matching the image illuminance score and the texture richness score; analyze the lidar point cloud to determine the average density of the lidar point cloud and the proportion of reflectivity anomalies in the lidar point cloud; determine a lidar effectiveness score matching the average density and the proportion; determine an ultra-wideband effectiveness score based on the inspection location and the signal-to-noise ratio corresponding to each of the base stations; and perform a weighted summation of the visual effectiveness score, the lidar effectiveness score, and the ultra-wideband effectiveness score to obtain the inspection reliability score.

[0151] Each module in the aforementioned inspection drone control device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in the processor of a computer device in hardware form or independent of it, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0152] In one exemplary embodiment, a computer device is provided, which may be a server, and its internal structure diagram may be as follows: Figure 6As shown, the computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and a database. The internal memory provides the environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The database stores data from the inspection and control process of the inspection drone. The I / O interfaces are used for information exchange between the processor and external devices. The communication interface is used for communication with external terminals via a network connection. When the computer program is executed by the processor, it implements an inspection and control method for an inspection drone.

[0153] Those skilled in the art will understand that Figure 6 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0154] In one embodiment, a computer device is also provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in the above method embodiments.

[0155] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements the steps in the above method embodiments.

[0156] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in the above method embodiments.

[0157] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0158] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0159] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0160] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A method for controlling the inspection of unmanned aerial vehicles (UAVs), characterized in that, The method includes: The inspection drone collects lidar point clouds and depth camera point clouds of the utility tunnel, as well as the inspection information of the drone. The point cloud consistency evaluation score is determined based on the lidar point cloud and the depth camera point cloud. The inspection reliability score is determined based on the lidar point cloud, the depth camera point cloud, and the inspection information. Based on the point cloud consistency evaluation score and the inspection reliability score, the current inspection trajectory is optimized to obtain an updated inspection trajectory. According to the updated inspection trajectory, the inspection drone is controlled to inspect the utility tunnel.

2. The method according to claim 1, characterized in that, The step of optimizing the current inspection trajectory based on the point cloud consistency evaluation score and the inspection reliability score to obtain an updated inspection trajectory includes: The point cloud consistency assessment score and the inspection reliability score are weighted and summed to obtain the fusion confidence score. Determine the dynamic safety distance threshold, gradient optimization step size, and dynamic weights that match the fusion confidence level; By inputting the dynamic safety distance threshold, the gradient optimization step size, and the dynamic weights into the cost function of the trajectory optimization planner, the current inspection trajectory is optimized to obtain the updated inspection trajectory output by the trajectory optimization planner.

3. The method according to claim 2, characterized in that, The step of optimizing the current inspection trajectory by inputting the dynamic safety distance threshold, the gradient optimization step size, and the dynamic weights into the cost function of the trajectory optimization planner to obtain the updated inspection trajectory output by the trajectory optimization planner includes: Obtain the fused point cloud corresponding to the lidar point cloud and the depth camera point cloud; Spatial construction is performed on the inspection 3D map corresponding to the fused point cloud to obtain the inspection spatial map; With the dynamic safety distance threshold, the gradient optimization step size, and the dynamic weights input into the cost function of the trajectory optimization planner, the current inspection trajectory is optimized based on the tunnel structure data and the inspection space map to obtain the updated inspection trajectory output by the trajectory optimization planner.

4. The method according to claim 2, characterized in that, The determination of the dynamic safety distance threshold, gradient optimization step size, and dynamic weights that match the fusion confidence includes: When the fusion confidence is greater than the first confidence threshold, the preset first safety distance threshold, the first gradient optimization step size, and the first dynamic weight are respectively determined as the dynamic safety distance threshold, gradient optimization step size, and dynamic weight that match the fusion confidence. When the fusion confidence is less than or equal to the first confidence threshold and greater than the second confidence threshold, the preset second safety distance threshold, the second gradient optimization step size, and the second dynamic weight are respectively determined as the dynamic safety distance threshold, the gradient optimization step size, and the dynamic weight that match the fusion confidence; the second safety distance threshold is greater than the first safety distance threshold, the second gradient optimization step size is less than the first gradient optimization step size, and the second dynamic weight is greater than the first dynamic weight; When the fusion confidence is less than or equal to the second confidence threshold, the preset third safety distance threshold, third gradient optimization step size, and third dynamic weight are respectively determined as the dynamic safety distance threshold, gradient optimization step size, and dynamic weight that match the fusion confidence; the third safety distance threshold is greater than the second safety distance threshold, the third gradient optimization step size is less than the second gradient optimization step size, and the third dynamic weight is greater than the second dynamic weight.

5. The method according to claim 2, characterized in that, The determination of the dynamic safety distance threshold, gradient optimization step size, and dynamic weights that match the fusion confidence includes: Based on the fusion confidence level, a dynamic safety distance threshold matching the fusion confidence level is obtained through a first parameter relationship between the preset fusion confidence level and the dynamic safety distance threshold. Based on the fusion confidence level, a gradient optimization step size matching the fusion confidence level is obtained through a second parameter relationship between the preset fusion confidence level and the gradient optimization step size. Based on the fusion confidence level, a dynamic weight matching the fusion confidence level is obtained through a third parameter relationship between the preset fusion confidence level and the dynamic weight.

6. The method according to any one of claims 1 to 5, characterized in that, The inspection information includes the inspection location; the utility tunnel is equipped with an ultra-wideband base station network, which includes multiple base stations; determining the inspection reliability score based on the lidar point cloud, the depth camera point cloud, and the inspection information includes: The image corresponding to the point cloud of the depth camera is analyzed to determine the image illumination score and texture richness score of the image. Determine a visual effectiveness score that matches the image illuminance score and the texture richness score; The lidar point cloud is analyzed to determine the average density of the lidar point cloud and the proportion of reflectivity anomalies in the lidar point cloud. Determine the effectiveness score of the lidar that matches the average density and the ratio; The ultra-wideband effectiveness score is determined based on the inspection location and the signal-to-noise ratio of each base station. The inspection reliability score is obtained by weighted summation of the visual effectiveness score, the lidar effectiveness score, and the ultra-wideband effectiveness score.

7. A patrol control device for an inspection drone, characterized in that, The device includes: The acquisition module is used to acquire the lidar point cloud and depth camera point cloud collected by the inspection drone on the pipe gallery, as well as the inspection information of the inspection drone. The first calculation module is used to determine the point cloud consistency evaluation score based on the lidar point cloud and the depth camera point cloud. The second calculation module is used to determine the inspection reliability score based on the lidar point cloud, the depth camera point cloud, and the inspection information. The processing module is used to optimize the current inspection trajectory based on the point cloud consistency evaluation score and the inspection reliability score to obtain an updated inspection trajectory. The control module is used to control the inspection drone to inspect the utility tunnel according to the updated inspection trajectory.

8. A computer device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images