Transformer substation indoor route planning method and system based on multi-view heterogeneous unmanned aerial vehicle

Abstract

The invention discloses a substation indoor track planning method and system based on a multi-view heterogeneous unmanned aerial vehicle, and aims to solve the problem of accurate track planning of an unmanned aerial vehicle under the conditions that no GPS exists in a substation and the environment is complex. The method comprises the following four steps: S1, acquiring an image, depth, infrared and IMU data acquired by an unmanned aerial vehicle, and constructing a database with indexes and version control through denoising, distortion removal, quality evaluation and semantic annotation preprocessing; s2, through feature alignment and time synchronization, generating unified feature representation for the governed data, and establishing a fusion model consistent with a space coordinate system; s3, predicting the panoramic depth map based on the unified features, and outputting an indoor dense map and the pose of the unmanned aerial vehicle in combination with optimization of a vision-inertial odometer and a factor graph; and S4, constructing a planning strategy through reinforcement learning and graph optimization to generate an optimal flight path, and performing online obstacle avoidance adjustment according to sensing feedback in flight. According to the invention, the automation degree, the operation safety and the system expansibility under the scenes of indoor inspection, emergency response and the like of the transformer substation can be obviously improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of power operation and maintenance, in particular to a substation indoor flight path planning method and system based on multi-view heterogeneous unmanned aerial vehicles. BACKGROUND

[0002] The existing substation inspection technology mainly includes manual inspection, track robot inspection and unmanned aerial vehicle inspection. Manual inspection relies on experienced operation and maintenance personnel to carry handheld devices for on-site inspection, but it is low in efficiency and high in risk, especially when working at high voltage or at night. Although track robots can achieve automation, they are limited by fixed tracks and cannot cover all corners of the substation, and the installation cost is high. As a new way, unmanned aerial vehicle inspection has the advantages of strong flexibility and wide coverage, but the current system has the following significant defects:

[0003] Firstly, the positioning accuracy is insufficient. Traditional unmanned aerial vehicles mostly use GPS or monocular vision SLAM. In the environment where indoor GPS signal is missing and visual features are sparse, the positioning error can reach meters, which cannot meet the requirement of centimeter-level accuracy of the substation. For example, in the cable tunnel, the single texture of the wall can cause feature matching failure, and then cause pose drift.

[0004] Secondly, the mapping and perception ability is weak. The existing system is difficult to fuse multi-source heterogeneous data such as RGB image, depth information, infrared thermal imaging and IMU inertial measurement, which leads to slow response to dynamic obstacles (such as moving personnel or equipment). At the same time, there is a lack of efficient data management mechanism, the collected raw data has large noise and inaccurate labeling, which affects the training effect of the deep learning model.

[0005] Thirdly, the flight path planning is static. Most systems use preset path planning, which cannot adapt to environmental changes in real time, such as sudden obstacles or sudden changes in light, which increases the risk of collision. In the multi-unmanned aerial vehicle cooperation scene, there is a lack of effective communication and map sharing mechanism, which easily causes path conflict or task redundancy.

[0006] Finally, the system has poor expandability. The existing scheme is mostly designed for special hardware, and the model deployment depends on high-performance servers, which cannot run in real time on edge devices, and the knowledge update cycle is long, which cannot quickly respond to changes in substation layout or addition of new equipment.

[0007] In view of the above problems, the present application proposes a new method and system, which forms a technical closed loop through data closed loop management, multi-modal fusion, deep learning mapping, dynamic planning and multi-machine cooperation, and improves the autonomy and reliability of the indoor unmanned aerial vehicle of the substation. SUMMARY

[0008] The purpose of the present application is to overcome the shortcomings of the prior art, provide a substation indoor flight path planning method and system based on multi-view heterogeneous unmanned aerial vehicles. The method realizes centimeter-level positioning, dynamic obstacle avoidance and efficient inspection of unmanned aerial vehicles in complex indoor environments by constructing a high-quality image database, realizing multi-source data fusion, training a deep learning model to generate a panoramic depth map, optimizing flight path planning strategy and multi-vehicle cooperative control.

[0009] To achieve the above purpose, the technical scheme adopted by the present application is as follows:

[0010] A substation indoor flight path planning method based on multi-view heterogeneous unmanned aerial vehicles, comprising the following steps:

[0011] S1: data management and database construction, acquiring images, depth, infrared and IMU data collected by multi-view heterogeneous unmanned aerial vehicles in the substation indoor, performing data preprocessing, denoising, distortion correction, quality evaluation and semantic labeling, and constructing a data database with index and version control;

[0012] S2: multi-source heterogeneous perception fusion, using feature alignment and time synchronization mechanism for the managed data, generating unified feature representation, and establishing a data fusion model consistent with the spatial coordinate system;

[0013] S3: deep learning mapping and positioning, panoramic depth map prediction based on unified feature representation, combined with visual-inertial odometer positioning and factor graph optimization to output indoor dense map and unmanned aerial vehicle pose;

[0014] S4: flight path planning and dynamic obstacle avoidance, constructing a flight path planning strategy based on reinforcement learning and graph optimization, generating an optimal flight path, and executing online obstacle avoidance adjustment according to sensor feedback during flight.

[0015] The S1 comprises the following steps:

[0016] S1.1 Multi-source data acquisition

[0017] The multi-view heterogeneous unmanned aerial vehicle is equipped with 4 types of perception devices:

[0018] High-definition optical camera: collect indoor equipment appearance image data , wherein is the pixel coordinate, is the acquisition timestamp;

[0019] Laser radar: collect depth data , is the three-dimensional space coordinate;

[0020] Infrared thermal imager: collect equipment temperature image , is the pixel corresponding temperature value;

[0021] Inertial Measurement Unit (IMU): Acquires UAV attitude data , It is the angular velocity vector. It is the angular acceleration vector;

[0022] S1.2 Data Preprocessing

[0023] Denoising: Gaussian filtering is used to remove image noise. The formula is:

[0024]

[0025] in For Gaussian kernel function, Standard deviation The width of the filter window is half its width.

[0026] The lidar point cloud is denoised using statistical filtering to remove outliers whose distance means exceed 3 times the standard deviation. The formula is as follows:

[0027] ,in For point cloud data, The mean of the local neighborhood points. The standard deviation of the neighborhood points;

[0028] Distortion Correction: For optical camera distortion, Zhang's calibration method is used, and the correction formula is as follows:

[0029]

[0030] in For the camera intrinsic parameter matrix ( , Focal length (Primary point coordinates) Let be a rotation matrix. It is a translation vector. These are the coordinates of the distorted pixels. The coordinates are after correction;

[0031] Quality assessment: Image sharpness is used. With point cloud integrity The dual-indicator evaluation uses the following formula:

[0032] ( (Image resolution);

[0033] For the effective number of point clouds, The total number of point clouds collected is required. ;

[0034] S1.3 Database Construction

[0035] The distributed database is established, the B + tree index structure is adopted, the index key is "equipment number - collection timestamp - data type", and a version control mechanism is introduced, and the version number is the basic version number, the data update interval, to ensure data traceability and consistency.

[0036] The S2 specifically includes the following steps:

[0037] S2.1 Feature alignment and time synchronization mechanism

[0038] Spatial alignment: taking the laser radar coordinate system as the reference, converting other device data to the coordinate system, and the conversion formula is:

[0039]

[0040] Among them, is the source coordinate system data, is the target coordinate system data, is the rotation matrix, which is obtained by iterative optimization through the ICP algorithm, and the iteration termination condition is , is the translation vector;

[0041] Time alignment: using linear interpolation method to compensate the time difference of different devices, assuming that the device A timestamp , the device B timestamp ( ), the data of device B at moment needs to be generated, and the formula is:

[0042]

[0043] Among them, is the adjacent two time stamps of device B, is the corresponding moment data.

[0044] S2.2 Data fusion model

[0045] A unified feature representation model based on weighted fusion is constructed, and the fusion formula is:

[0046]

[0047] Among them, are the feature vectors of optical, laser radar, infrared and IMU data respectively; the weight is calculated by entropy weight method, and the formula is:

[0048] , For the first class data information entropy, is the number of samples, is the first class data The feature proportion of the first sample, and satisfies

[0049] The S3 specifically comprises the following steps:

[0050] S3.1 Panoptic depth map prediction

[0051] An improved U-Net network model is adopted, the input is a unified feature , and the output is a panoramic depth map , the network loss function adopts L1 loss and structural similarity loss (SSIM) weighting, the formula is:

[0052]

[0053] Wherein is the measured depth of laser radar (Ground Truth), is a weight coefficient, is the mean of , is the standard deviation, is the covariance, is a constant;

[0054] S3.2 Visual-inertial odometry

[0055] Combined with IMU data and visual features, the pose of the unmanned aerial vehicle is calculated , wherein is a rotation matrix, is a translation vector, and the calculation process satisfies:

[0056] ,

[0057] Wherein is the velocity vector of the unmanned aerial vehicle, is the measured acceleration of the IMU, is the gravity acceleration vector; at the same time, the pose is optimized through visual feature matching, the feature matching adopts FLANN algorithm, and the matching error threshold is set to is the Euclidean distance of the feature points.

[0058] S3.3 Factor graph optimization

[0059] ​​​​Constructing factor graph where nodes include UAV pose nodes ( ) and IMU pre-integration nodes , edges include visual observation factors , IMU pre-integration factors , and the optimization objective is to minimize global error:

[0060]

[0061] where is the visual observation value, are the covariance matrices of visual and IMU factors respectively, and the Gauss-Newton method is used for iterative solution with the iteration termination condition , and the final output is an indoor dense map and an optimal UAV pose .

[0062] The S4 includes the following steps:

[0063] S4.1 Reinforcement learning path planning strategy

[0064] A deep reinforcement learning (DRL) model is used, and the state space is defined as the working constraints, including the maximum flight speed , the minimum turning radius , the action space is the speed control quantity, is the angular velocity control quantity, and the reward function is designed as:

[0065]

[0066] where , the reward for reaching the target point, is the distance between the UAV and the obstacle, is the indicator function, triggering the collision penalty, is the distance flown, and the redundant path is penalized; the network uses a DQN architecture, the experience replay pool capacity is set to , the target network update interval is 1000 steps, and the learning rate ;

[0067] S4.2 Graph optimization path smoothing

[0068] The initial path output by the reinforcement learning ( ) is input into the graph optimization model, and the optimization objective is:

[0069]

[0070] The first term is the path length penalty, and the second term is the smoothness penalty. To smooth out the weights, a quadratic programming approach is used to obtain the optimal trajectory. ;

[0071] S4.3 Dynamic Obstacle Avoidance Control

[0072] Real-time acquisition of obstacle dynamic information fed back by sensors For the first The position and velocity of each obstacle are used to adjust the trajectory using the artificial potential field method, and the gravitational force acting on the UAV is also considered. Repulsive force The formula is:

[0073] ( The gravitational coefficient, For the target point, (This refers to the current location of the drone).

[0074] ( The repulsion coefficient is... For a safe distance, For drones and the first (real-time distance of each obstacle)

[0075] Overall control ,according to Adjusting the speed and angular velocity of the drone enables dynamic obstacle avoidance.

[0076] It also includes the S5 step: multi-drone collaboration and safety control. Through multi-drone map sharing and task allocation, it enables path interlocking and safety redundancy control when multiple drones are conducting collaborative inspections in a substation. Specifically, it includes:

[0077] S5.1 Multi-machine map sharing

[0078] Adopting a communication architecture that combines 5G private networks with edge computing, the communication speed... Delay Data transmission uses the TCP / IP protocol and incorporates a redundancy check mechanism. The check formula is:

[0079] For the first One data byte, For checksum verification, the receiving end verifies... Consistency is required; if inconsistent, retransmission is necessary.

[0080] S5.2 Task Assignment

[0081] An improved genetic algorithm is used for task allocation, and the objective function is to minimize the total inspection time , The task time of the first The constraint condition is that the task amount of each unmanned aerial vehicle The maximum task amount of a single unmanned aerial vehicle, and the algorithm steps include:

[0082] Encoding: integer encoding is used, and the chromosome The unmanned aerial vehicle number allocated to the first task;

[0083] Selection: roulette wheel selection is used, and the selection probability , The total time corresponding to the first chromosome, The population size;

[0084] Crossover and mutation: the crossover probability , and the mutation probability ;

[0085] Iteration termination: when the iteration number or the total time difference between adjacent two generations , the optimal task allocation scheme is output.

[0086] S 5.3 Path interlocking and safety redundancy

[0087] A path interlocking mechanism is established, and the flight path of the unmanned aerial vehicle The flight path of the unmanned aerial vehicle , and the interlocking condition is that at any moment , The safety distance of multiple machines , If violated, path adjustment is triggered; Safety redundancy control adopts a "primary-backup" mechanism. When the primary machine fails (the judgment condition is that the communication is interrupted for 3 times in a row or the attitude error

[0088] ), the backup machine immediately takes over the task, and the switching time , ensuring that the inspection is not interrupted. It also includes model lightening and deployment steps. Through mixed precision training, model pruning and quantization, the depth estimation model is deployed to Jetson or RK series edge computing platforms, realizing single-frame inference delay less than 200 milliseconds.

[0089] A substation indoor flight path planning system based on multi-view heterogeneous unmanned aerial vehicles, comprising:

[0090]

[0091] ​A data governance and database construction module is configured to acquire images, depth, infrared and IMU data collected by multi-view heterogeneous UAVs in a substation indoor, perform preprocessing operations of denoising, distortion removal, quality assessment and semantic labeling on the data, and construct a data database with indexing mechanism and version control capability.

[0092] A multi-source heterogeneous perception fusion module is communicatively connected to the data governance and database construction module, configured to perform time synchronization and feature alignment processing on the preprocessed data, generate a unified feature representation, and establish a multi-source data fusion model consistent with a preset spatial coordinate system.

[0093] A deep learning mapping and positioning module is communicatively connected to the multi-source heterogeneous perception fusion module, configured to predict a panoramic depth map based on the unified feature representation, perform pose estimation in combination with a visual-inertial odometer, and output high-precision indoor dense three-dimensional map and real-time UAV pose information through a factor graph optimization algorithm.

[0094] A flight path planning and dynamic obstacle avoidance module is communicatively connected to the deep learning mapping and positioning module, configured to construct a flight path planning model that integrates reinforcement learning strategy and graph optimization method based on the dense map and pose information, generate an optimal flight path from a starting point to a target point, and perform online dynamic obstacle avoidance adjustment according to real-time sensor feedback during UAV flight.

[0095] The multi-view heterogeneous UAVs include at least two types of UAVs carrying visible light cameras, depth sensors, infrared thermal imagers and inertial measurement units (IMUs), and each sensor data has spatiotemporal consistency calibration parameters.

[0096] The system is deployed on a cloud server or an edge computing node, and cooperates with a multi-view heterogeneous UAV group through wireless communication to realize autonomous navigation and precise positioning of a substation indoor inspection task.

[0097] The innovations of the present application are as follows:

[0098] 1. Innovative construction of multi-dimensional data governance system

[0099] Break through the limitations of traditional UAV data processing "single, no traceability", innovatively integrate data governance concepts into indoor substation scenarios. For multi-view heterogeneous UAV collected images, depth, infrared and IMU multi-source data, establish a "pre-processing - quality control - semantic labeling - version management" full-link governance mechanism: through denoising, distortion correction algorithm to solve the data distortion problem caused by indoor complex lighting (such as equipment shadow, light reflection) and electromagnetic environment; Introducing a quality assessment module to filter out invalid data and avoid interference from low-quality data in subsequent processes; Semantic labeling combined with substation equipment characteristics (such as switch cabinet, bus, insulator) to build a dedicated tag system, laying the foundation for accurate perception; And the database with index and version control realizes data traceability and reusability, solves the data consistency problem when multiple machines cooperate or repeat inspection, and fills the technical gap in indoor substation UAV data management.

[0100] 2. Multi-source heterogeneous perception fusion and collaborative optimization innovation

[0101] In view of the heterogeneity of image, depth, infrared, IMU data (spatial and temporal dimensions, data type differences), the innovation adopts "feature alignment + time synchronization" dual mechanism to realize data fusion. Through the feature alignment algorithm, the common features of different modal data (such as device outline, spatial position) are extracted to generate a unified feature representation; The time synchronization mechanism solves the multi-sensor data acquisition delay problem, ensures the accurate matching of data and spatial coordinate system, and builds a "spatial-temporal consistency, multi-modal complementarity" data fusion model. This innovation breaks through the limitations of traditional single-mode perception, realizing all-around, high-precision perception of indoor substation environment.

[0102] 3. Deep learning driven mapping and positioning integration innovation

[0103] Abandoning the traditional separation of mapping and positioning, the innovation combines panoramic depth map prediction with visual-inertial odometry (VIO) and integrates factor graph optimization technology to realize mapping and positioning integration. Based on unified feature representation, the deep learning model accurately predicts the panoramic depth map, restoring the complex spatial structure of the indoor environment (such as narrow passages, equipment-intensive areas); VIO technology takes full advantage of the complementarity of image visual information and IMU inertial data to realize real-time solution of UAV pose; Factor graph optimization globally corrects errors in the mapping and positioning process, effectively reducing cumulative errors. This innovation solves the problem of high-precision positioning and dense mapping in indoor substations without GPS, providing reliable environmental model support for path planning.

[0104] 4. Reinforcement learning and graph optimization integrated dynamic path planning innovation

[0105] An innovative "reinforcement learning + graph optimization" dual-driven path planning strategy is proposed, which takes into account both global optimization and dynamic adaptability. Through the reinforcement learning algorithm, the indoor substation inspection task requirements (such as full coverage, obstacle avoidance, and efficiency) are used as the reward function to train and generate globally optimal paths. Meanwhile, graph optimization technology is introduced to smooth and optimize the path, ensuring the stability of the UAV flight. For dynamic changes in the indoor environment (such as temporary obstacles and equipment state changes), an online obstacle avoidance adjustment mechanism is designed to dynamically update the path based on real-time sensor feedback. This innovation breaks through the limitations of traditional fixed path planning, achieving the dual goals of "global optimization + dynamic adaptation", and improving the flexibility and reliability of path planning.

[0106] Advantages and effects of the present application:

[0107] 1. Efficient and reliable data processing lays the foundation for high-quality perception

[0108] The construction of the data governance system improves the utilization rate of multi-source heterogeneous data by more than 30%, and the accuracy of denoised and deformed data is more than 95%. Semantic annotation provides accurate semantic information for subsequent perception and mapping. The versioned database supports multi-machine cooperative inspection and historical data comparison and analysis, reducing data management costs and providing complete data support for substation equipment state evaluation.

[0109] 2. Accurate and comprehensive perception fusion improves environmental awareness

[0110] The multi-source heterogeneous perception fusion model realizes the complementary advantages of different modal data. Infrared data can accurately identify equipment thermal defects, depth data restores spatial structure, image data captures equipment appearance details, and IMU data ensures motion state perception. Compared with single-mode perception, environmental perception accuracy is improved by more than 40%, effectively identifying hidden equipment defects and spatial obstacles in substations, providing comprehensive environmental information for subsequent mapping and positioning and path planning.

[0111] 3. High-precision mapping and positioning adapt to complex indoor environments

[0112] The integrated mapping and positioning technology makes the resolution of indoor dense maps reach centimeter level, and the UAV pose positioning error is controlled within 5cm, meeting the high-precision navigation requirements of indoor narrow spaces and equipment-intensive areas in substations. Factor graph optimization effectively suppresses cumulative error, ensuring stable positioning accuracy even in long-time flight scenarios, solving the error accumulation pain point of traditional methods in GPS-free environments, and ensuring that the UAV does not get lost or collide in complex indoor environments.

[0113] 4. Optimal and flexible path planning ensures efficient and safe inspection

[0114] The trajectory planning strategy fuses reinforcement learning and graph optimization, so that the generated trajectory meets the "shortest path + full coverage + no collision" inspection requirements, the inspection efficiency is improved by more than 25%, and the flight time and energy consumption of the unmanned aerial vehicle are reduced. The response time of the online obstacle avoidance adjustment mechanism is less than 0.5 seconds, which can quickly avoid temporary obstacles, and the flight safety rate is more than 99.9%. The technology enables the unmanned aerial vehicle to autonomously complete the inspection task in the complex indoor area of the transformer substation, reduces manual intervention, and reduces the safety risk of manual inspection (such as high-altitude operation and electromagnetic radiation exposure).

[0115] 5. Adapt to the characteristics of the transformer substation scene, and the practicality is strong

[0116] The scheme fully considers the particularity of the indoor environment of the transformer substation (no GPS, narrow space, dense equipment, and strong electromagnetic interference), and each link technology is optimized and designed for this scene, so the adaptability is strong. Compared with the general trajectory planning method, the application success rate in the indoor scene of the transformer substation is improved by more than 50%, which can be widely applied to indoor inspection of transformer substations of 35kV and above voltage levels, realizes accurate identification of equipment defects, and comprehensive monitoring of the space environment, provides technical support for safe operation of the transformer substation, and has significant economic and social benefits. BRIEF DESCRIPTION OF DRAWINGS

[0117] Figure 1 It is a schematic diagram of the overall process of the method of the present application. Figure 2 It is a schematic diagram of the multi-unmanned aerial vehicle cooperative trajectory planning scene. DETAILED DESCRIPTION

[0118] The present application will be described in detail below in conjunction with the drawings and specific embodiments. It should be understood that these embodiments are only used to illustrate the present application, and not to limit its scope. Those skilled in the art can make various modifications or equivalent replacements according to the teaching of the present application, which all fall within the protection scope of the present application.

[0119] Reference Figure 1 , Figure 2 .

[0120] A transformer substation indoor trajectory planning method based on multi-view heterogeneous unmanned aerial vehicles includes the following steps:

[0121] S1: Data governance and database construction, obtain the images, depth, infrared and IMU data collected by multi-view heterogeneous unmanned aerial vehicles in the transformer substation, perform data preprocessing, denoising, distortion correction, quality assessment and semantic labeling, and construct a data database with indexing and version control;

[0122] S2: Multi-source heterogeneous perception fusion, using feature alignment and time synchronization mechanism for the governed data, generating a unified feature representation, and establishing a data fusion model consistent with the spatial coordinate system;

[0123] S3: Deep learning mapping and localization, panoramic depth map prediction based on unified feature representation, combined with visual-inertial odometry positioning and factor graph optimization to output indoor dense map and UAV pose;

[0124] S4: Trajectory planning and dynamic obstacle avoidance, build trajectory planning strategy based on reinforcement learning and graph optimization, generate optimal trajectory, and execute online obstacle avoidance adjustment according to sensor feedback during flight.

[0125] The S1 includes the following steps:

[0126] S1.1 Multi-source data acquisition

[0127] Multi-view heterogeneous UAV carries 4 types of perception devices:

[0128] High-definition optical camera: collect indoor device appearance image data , wherein is the pixel coordinate, is the acquisition timestamp;

[0129] Laser radar: collect depth data , is the three-dimensional space coordinate;

[0130] Infrared thermal imager: collect device temperature image , is the pixel corresponding temperature value;

[0131] Inertial measurement unit (IMU): collect UAV attitude data , is the angular velocity vector, is the angular acceleration vector;

[0132] S1.2 Data preprocessing

[0133] Noise removal: Gaussian filter is used to remove image noise, the formula is:

[0134]

[0135] wherein is the Gaussian kernel function, is the standard deviation, is the filter window half-width;

[0136] Statistical filtering is used for laser radar point cloud denoising, and abnormal points with distance mean exceeding 3 times the standard deviation are removed, the formula is:

[0137] wherein is the point cloud data, Mean of local neighborhood points, Standard deviation of neighborhood points;

[0138] De-distortion: Zhang's calibration method is used to correct optical camera distortion, and the correction formula is:

[0139]

[0140] Wherein is the camera intrinsic matrix ( , is the focal length, is the principal point coordinate), is the rotation matrix, is the translation vector, is the distorted pixel coordinate, is the corrected coordinate;

[0141] Quality assessment: image sharpness and point cloud integrity are used as double indicators for evaluation, and the formula is:

[0142] ( is the image resolution);

[0143] is the effective point cloud number, is the total number of collected point clouds, and the requirement ;

[0144] S1.3 Database construction

[0145] A distributed database is established, using a B + tree index structure, and the index key is "device number - collection timestamp - data type". At the same time, a version control mechanism is introduced, and the version number is the base version number, is the data update interval, ensuring data traceability and consistency.

[0146] The S2 specifically includes the following steps:

[0147] S2.1 Feature alignment and time synchronization mechanism

[0148] Spatial alignment: taking the laser radar coordinate system as the reference, the data of other devices is converted to this coordinate system, and the conversion formula is:

[0149]

[0150] Wherein is the source coordinate system data, is the target coordinate system data, is the rotation matrix, which is obtained by iterative optimization through ICP algorithm, and the iteration termination condition is​ , It is a translation vector;

[0151] Time alignment: Linear interpolation is used to compensate for time differences between different devices. Let device A have a timestamp. Device B timestamp ( ), needs to be generated Data from time device B The formula is:

[0152]

[0153] in For two adjacent timestamps of device B, This is the data for the corresponding time point.

[0154] S2.2 Data Fusion Model

[0155] A unified feature representation model based on weighted fusion is constructed, and the fusion formula is as follows:

[0156]

[0157] in These are the feature vectors of optical, lidar, infrared, and IMU data, respectively; weights. The formula, calculated using the entropy weight method, is as follows:

[0158] , ( For the first Information entropy of class data, For the sample size, For the first Class Data (the feature proportion of each sample), and satisfying .

[0159] S3 specifically includes the following steps:

[0160] S3.1 Panoramic Depth Map Prediction

[0161] An improved U-Net network model is used, with uniform features as input. The output is a panoramic depth map. The network loss function uses a weighted average of L1 loss and structural similarity loss (SSIM), as shown in the formula:

[0162]

[0163] in This refers to the ground truth, measured by lidar. These are the weighting coefficients. ( for The mean, Standard deviation For covariance, (where the constant is)

[0164] S3.2 Vision-Inertial Odometry Positioning

[0165] By combining IMU data with visual features, the pose of the drone can be calculated. ,in Let be a rotation matrix. As a translation vector, the calculation process satisfies:

[0166] ,

[0167] in For the drone's velocity vector, For measuring acceleration with an IMU, This represents the gravitational acceleration vector; simultaneously, pose is optimized through visual feature matching, using the FLANN algorithm, with a matching error threshold set to... ( (Euclidean distance between feature points).

[0168] S3.3 Factor Plot Optimization

[0169] Constructing factor graphs , where nodes Including UAV pose nodes ( ) and IMU pre-integration node ,side Including visual observation factors IMU pre-integration factor The optimization objective is to minimize the global error:

[0170]

[0171] in These are visual observations. The covariance matrices of the visual and IMU factors are given, respectively. The Gauss-Newton method is used for iterative solution, with the iteration termination condition being... The final output is a dense indoor map. Optimal pose of the drone .

[0172] S4 includes the following steps:

[0173] S4.1 Reinforcement Learning Path Planning Strategy

[0174] A deep reinforcement learning (DRL) model is used to define the state space. Operational constraints, including maximum flight speed Minimum turning radius Action space For speed control, Angular velocity control quantity, reward function Designed as follows:

[0175]

[0176] in Rewards for reaching the target point. The distance between the drone and the obstacle. For indicator functions, Collision penalty is triggered at times. The distance already flown is used to penalize redundant paths; the network adopts a DQN architecture, and the experience replay pool capacity is set to [value missing]. The target network update interval is 1000 steps, and the learning rate is... ;

[0177] S4.2 Optimized Track Smoothing

[0178] The initial trajectory output by reinforcement learning ( The input graph optimization model has the following optimization objective:

[0179]

[0180] The first term is the path length penalty, and the second term is the smoothness penalty. To smooth out the weights, a quadratic programming approach is used to obtain the optimal trajectory. ;

[0181] S4.3 Dynamic Obstacle Avoidance Control

[0182] Real-time acquisition of obstacle dynamic information fed back by sensors For the first The position and velocity of each obstacle are used to adjust the trajectory using the artificial potential field method, and the gravitational force acting on the UAV is also considered. Repulsive force The formula is:

[0183] ( The gravitational coefficient, For the target point, (This refers to the current location of the drone).

[0184] ( The repulsion coefficient is... For a safe distance, For drones and the first (real-time distance of each obstacle)

[0185] Overall control ,according to Adjusting the speed and angular velocity of the drone enables dynamic obstacle avoidance.

[0186] It also includes the S5 step: multi-drone collaboration and safety control. Through multi-drone map sharing and task allocation, it enables path interlocking and safety redundancy control when multiple drones are conducting collaborative inspections in a substation. Specifically, it includes:

[0187] S5.1 Multi-machine map sharing

[0188] Adopting a communication architecture that combines 5G private networks with edge computing, the communication speed... Delay Data transmission uses the TCP / IP protocol and incorporates a redundancy check mechanism. The check formula is:

[0189] For the first One data byte, For checksum verification, the receiving end verifies... Consistency is required; if inconsistent, retransmission is necessary.

[0190] S5.2 Task Assignment

[0191] An improved genetic algorithm is used for task allocation, with the objective function being to minimize the total inspection time. , For the first The mission time for each drone is constrained by the mission volume per drone. To determine the maximum workload for a single drone, the algorithm steps include:

[0192] Encoding: Integer encoding is used for chromosomes. For the first The drone number assigned to each task;

[0193] Selection: A roulette wheel selection method is used, with selection probability... , For the first The total time corresponding to each chromosome Population size;

[0194] Crossover and Mutation: Crossover Probability Probability of mutation ;

[0195] Iteration termination: When the number of iterations reaches a certain threshold. Or the total time difference between two adjacent generations The process terminates at the appropriate time, and the optimal task allocation scheme is output.

[0196] S5.3 Path Interlocking and Safety Redundancy

[0197] Establish a path interlocking mechanism and deploy drones. flight path drones flight path The interlock condition is at any time , ( (to maintain safe distance between multiple machines); if this is violated, path adjustment will be triggered.

[0198] Safety redundancy control adopts a "master-slave" mechanism. When the master fails (the judgment condition is three consecutive communication interruptions or attitude error), the redundancy is automatically resolved. When this happens, the backup machine immediately takes over the task, switching time. To ensure uninterrupted inspections.

[0199] The substation indoor trajectory planning method based on multi-camera heterogeneous UAVs also includes model lightweighting and deployment steps. Each step involves hybrid precision training, model pruning and quantization to deploy the depth estimation model to Jetson or RK series edge computing platforms, achieving a single-frame inference latency of less than 200 milliseconds.

[0200] The features of the technical solution of this invention are as follows:

[0201] High-precision positioning and mapping: Integrating multi-view heterogeneous sensors and deep learning, it achieves dense depth map generation and factor map optimization, with positioning errors controlled within 5 cm, far superior to traditional SLAM methods. Strong dynamic adaptability: The trajectory planning combines reinforcement learning and graph optimization, supporting real-time replanning to handle dynamic scenarios such as personnel movement or equipment changes in substations. Data quality assurance: Introducing image cleaning and annotation and database management ensures high-quality and traceable training data, supporting continuous model iteration. Highly efficient multi-drone collaboration: Based on the ROS2 communication framework, it enables map sharing and task allocation, improving overall inspection efficiency by over 30%. Engineering-friendly: The lightweight model design supports edge deployment, has low inference latency, is suitable for various UAV platforms, and has good scalability, allowing migration to other indoor industrial scenarios.

[0202] Example 2:

[0203] A substation indoor trajectory planning system based on multi-camera heterogeneous unmanned aerial vehicles (UAVs) includes:

[0204] The data governance and database construction module is used to acquire image, depth, infrared and IMU data collected by multi-camera heterogeneous UAVs in the indoor substation, perform preprocessing operations such as denoising, distortion correction, quality assessment and semantic annotation on the data, and build a database with indexing mechanism and version control capabilities.

[0205] The multi-source heterogeneous sensing fusion module is communicatively connected to the data governance and database construction module. It is used to perform time synchronization and feature alignment processing on the preprocessed data, generate a unified feature representation, and establish a multi-source data fusion model consistent with the preset spatial coordinate system.

[0206] The deep learning mapping and localization module is communicatively connected to the multi-source heterogeneous perception fusion module. It is used to predict the panoramic depth map based on the unified feature representation, perform pose estimation by combining visual-inertial odometry, and output a high-precision indoor dense 3D map and real-time pose information of the UAV through a factor graph optimization algorithm.

[0207] The trajectory planning and dynamic obstacle avoidance module is communicatively connected to the deep learning mapping and localization module. It is used to construct a trajectory planning model that integrates reinforcement learning strategies and graph optimization methods based on the dense map and pose information, generate the optimal flight trajectory from the starting point to the target point, and perform online dynamic obstacle avoidance adjustments based on real-time sensor feedback during the UAV's flight.

[0208] Multi-camera heterogeneous UAVs include at least two types of UAVs equipped with visible light cameras, depth sensors, infrared thermal imagers, and inertial measurement units (IMUs), and the data from each sensor has spatiotemporal consistency calibration parameters.

[0209] The system is deployed on a cloud server or edge computing node and works in collaboration with a multi-camera heterogeneous drone swarm via wireless communication to achieve autonomous navigation and precise positioning for indoor inspection tasks in substations.

[0210] Example 3:

[0211] 1. Inspection Positioning and Planning of Single-UAV Substation Control Room

[0212] In this embodiment, a multi-camera heterogeneous UAV equipped with an RGB camera, a ToF depth sensor, an infrared camera, and an IMU is used to perform inspection tasks in the substation control room. The system first performs a data governance step: the UAV collects image data along a preset path and transmits it to the backend server. The data governance module uses the OpenCV library for denoising and distortion correction, evaluates image quality using the NIMA model, and retains only images with scores above a threshold. Subsequently, the Label Studio platform combined with the Mask R-CNN model is used for semi-automatic semantic annotation, identifying entities such as control cabinets, displays, and cables, and labeling them as "static equipment" or "potential obstacles." The labeled data is stored along with the original images in a MongoDB database, supporting version control and fast indexing. For example, for a batch of 10,000 collected images, the effective data rate after governance reaches 95%, and the annotation accuracy reaches 98%. Next, the multi-source perception fusion module is activated. The acquired data includes RGB images (1920x1080 resolution), ToF depth maps (640x480 resolution), infrared thermal images (320x240 resolution), and IMU six-axis data (100Hz sampling rate). The fusion process first achieves hardware-level time synchronization via the PTP protocol, with errors controlled within 1ms. Then, all data is projected onto a unified coordinate system using a pre-calibrated extrinsic matrix (obtained through checkerboard calibration). Feature extraction employs a Transformer encoder, performing convolutional extraction on RGB features and upsampling and fusing depth and infrared features, ultimately outputting a unified feature tensor with dimensions [batch, channel, height, width]. This fusion improves the system's robustness in low-light conditions; for example, in the event of a control room lighting failure, the infrared modality can compensate for the loss of RGB data.

[0213] The deep learning mapping and localization module receives unified features and inputs them into a custom depth estimation network. This network uses ResNet-50 as its backbone and integrates a Transformer attention module to capture long-range dependencies. During training, a self-built database is used in conjunction with the ScanNet dataset, and the loss function combines L1 depth loss and semantic consistency loss. In the inference phase, after generating a panoramic depth map, the initial pose is calculated using the visual-inertial odometry of the ORB-SLAM3 framework, and then input into the g2o factor map optimizer. IMU constraints and loop closure detection are added, and the optimized pose and dense map are output. In the experiments, the average localization error was 3.2 cm, and the mapping density reached 5000 points per cubic meter.

[0214] The trajectory planning module operates based on the generated map and localization results. Planning employs a DQN reinforcement learning strategy. The state space includes the current pose, map semantic layers (equipment area, passageway area, no-fly zone), and energy consumption estimation. The action space consists of velocity and direction adjustments. Training is conducted in the Gazebo simulation environment, with the reward function considering coverage, obstacle avoidance success rate, and energy consumption. During the online phase, graph optimization solves for smooth trajectories, ensuring path continuity. If a dynamic obstacle is detected (e.g., maintenance personnel entering), the module uses YOLOv8 to identify and update the no-fly zone in real time, triggering replanning with a response time of less than 500ms.

[0215] Finally, in this embodiment, the multi-machine collaborative module is simplified to a single-machine mode, but safety control logic is retained, such as automatic hovering or returning to the starting point when the pose drift exceeds the threshold. The entire process runs on the Jetson Xavier NX edge board with an end-to-end latency of 150ms, meeting real-time requirements.

[0216] 2. Multi-UAV Collaborative Inspection of Cable Tunnels: This embodiment extends to a multi-UAV scenario, where three heterogeneous UAVs (one quadcopter, one fixed-wing hybrid, and one tracked auxiliary) collaboratively inspect cable tunnels in substations. The data governance steps are similar to those in Embodiment 1, but with the addition of multi-UAV data merging: After each UAV uploads data, the governance module aligns the flight ID with the timestamp to build a global database, supporting cross-UAV version comparison. For example, if high noise is detected in the data from one UAV, it automatically supplements the data from other UAVs.

[0217] Multi-source sensing fusion is extended to a distributed model: each drone uploads its feature vectors after local fusion, and a central server performs global alignment, using a graph neural network (GNN) to model the multi-drone topology and ensure feature consistency. In the narrow environment of tunnels, this fusion significantly improves the detection accuracy of abnormal heat sources (such as overheated cables), with infrared data weights dynamically adjusted.

[0218] The deep learning-based mapping module enables multi-machine map fusion: after each machine generates a local depth map, they are incrementally merged using the OctoMap framework to resolve conflicts caused by parallax. Localization optimization uses a distributed factor map, with each machine maintaining a local sub-map and periodically synchronizing global constraints. Experiments show that in multi-machine mode, map integrity is improved by 20%, and the localization error is reduced to 2.5cm.

[0219] The trajectory planning employs Multi-Agent Reinforcement Learning (MARL), where each aircraft's policy network shares an experience pool, and communication latency is simulated during training. During planning, the task scheduling unit allocates sub-regions based on tunnel length, and paths are interlocked to avoid collisions. Dynamic obstacle avoidance is extended to multi-aircraft collaboration: when one aircraft detects an obstacle, it broadcasts an update, and all aircraft re-plan their paths.

[0220] Multi-machine collaborative control is the focus of this embodiment: based on the ROS2 DDS network, map synchronization (once per second) and task redistribution are achieved. The communication management unit monitors signal strength, switching to local buffer mode when signal quality is low, storing data to be uploaded after recovery. Fault redundancy includes: when one machine fails, other machines take over its area, predicting topology changes and adjusting strategies through GNN. In actual tunnel testing, the entire system achieved 98% coverage, and the collaborative efficiency was 50% higher than that of a single machine.

[0221] 3. Model Optimization and Edge Deployment: To adapt to resource-constrained UAV platforms, this embodiment focuses on model lightweighting. The initial parameter count of the deep estimation network was 50M, which was compressed to 15M through TensorRT quantization (INT8) and pruning, reducing the inference speed from 300ms to 120ms. Deployment steps included: exporting the ONNX model, compiling the engine on the Jetson platform, and supporting multi-threaded parallelism. Mixed-precision training (FP16) further accelerated convergence, and the training dataset was expanded to 200,000 substation images.

[0222] The system also integrates a visualization interface: using RViz to display real-time maps and flight paths, users can remotely monitor and intervene. Anomaly handling includes low battery return, communication loss hovering, etc., ensuring safety.

[0223] Through the above embodiments, this invention demonstrates its practicality and scalability in complex indoor substation environments. Future development could further integrate 5G communication and edge AI chips to improve response speed and autonomy.

Claims

1. A method for indoor flight path planning in substations based on multi-camera heterogeneous unmanned aerial vehicles (UAVs), characterized in that, Includes the following steps: S1: Data governance and database construction: acquire image, depth, infrared and IMU data collected by multi-camera heterogeneous UAVs in the indoor substation, perform data preprocessing, denoise removal, distortion removal, quality assessment and semantic annotation, and build a database with index and version control. S2: Multi-source heterogeneous sensing fusion utilizes feature alignment and time synchronization mechanisms to generate unified feature representations for the treated data and establishes a data fusion model consistent with the spatial coordinate system; S3: Deep learning mapping and localization, based on unified feature representation to predict panoramic depth maps, combined with visual-inertial odometry localization and factor map optimization to output dense indoor maps and UAV poses; S4: Path planning and dynamic obstacle avoidance. Construct a path planning strategy based on reinforcement learning and graph optimization to generate the optimal path and perform online obstacle avoidance adjustments based on sensor feedback during flight.

2. The substation indoor trajectory planning method based on multi-camera heterogeneous UAVs according to claim 1, characterized in that, S1 includes the following steps: S1.1 Multi-source data acquisition Multi-camera heterogeneous drones are equipped with four types of sensing devices: High-definition optical camera: Captures image data of the exterior of indoor equipment. ,in For pixel coordinates, For collection timestamps; LiDAR: Acquiring depth data , Three-dimensional spatial coordinates; Infrared thermal imager: Acquires temperature images of equipment. , The temperature value corresponding to the pixel; Inertial Measurement Unit (IMU): Acquires UAV attitude data , It is the angular velocity vector. It is the angular acceleration vector; S1.2 Data Preprocessing Denoising: Gaussian filtering is used to remove image noise. The formula is: ， in For Gaussian kernel function, Standard deviation The width of the filter window is half its width. The lidar point cloud is denoised using statistical filtering to remove outliers whose distance means exceed 3 times the standard deviation. The formula is as follows: ,in For point cloud data, The mean of the local neighborhood points. The standard deviation of the neighborhood points; Distortion Correction: For optical camera distortion, Zhang's calibration method is used, and the correction formula is as follows: ， in For the camera intrinsic parameter matrix ( , Focal length (Primary point coordinates) Let be a rotation matrix. It is a translation vector. These are the coordinates of the distorted pixels. The coordinates are after correction; Quality assessment: Image sharpness is used. With point cloud integrity The dual-indicator evaluation uses the following formula: ( (Image resolution); For the effective number of point clouds, The total number of point clouds collected is required. ; S1.3 Database Construction A distributed database is established, employing a B+ tree index structure with the index key being "device number - collection timestamp - data type". A version control mechanism is also introduced, with version numbers... base version number, Set data update intervals to ensure data traceability and consistency.

3. The substation indoor trajectory planning method based on multi-camera heterogeneous UAVs according to claim 1, characterized in that, S2 specifically includes the following steps: S2.1 Feature Alignment and Time Synchronization Mechanism Spatial alignment: Using the lidar coordinate system as a reference, data from other devices is transformed to this coordinate system. The transformation formula is as follows: ， in For source coordinate system data, For target coordinate system data, The rotation matrix is ​​obtained through iterative optimization using the ICP algorithm, with the iteration termination condition being... , It is a translation vector; Time alignment: Linear interpolation is used to compensate for time differences between different devices. Let device A have a timestamp. Device B timestamp ( ), needs to be generated Data from time device B The formula is: ， in For two adjacent timestamps of device B, Data for the corresponding time point; S2.2 Data Fusion Model A unified feature representation model based on weighted fusion is constructed, and the fusion formula is as follows: ， in These are the feature vectors of optical, lidar, infrared, and IMU data, respectively; weights. The formula, calculated using the entropy weight method, is as follows: , ( For the first Information entropy of class data, For the sample size, For the first Class Data (the feature proportion of each sample), and satisfying .

4. The method according to claim 1, characterized in that, S3 specifically includes the following steps: S3.1 Panoramic Depth Map Prediction An improved U-Net network model is used, with uniform features as input. The output is a panoramic depth map. The network loss function uses a weighted average of L1 loss and structural similarity loss (SSIM), as shown in the formula: ， in This refers to the ground truth, measured by lidar. These are the weighting coefficients. ( for The mean, Standard deviation For covariance, (where the constant is) S3.2 Vision-Inertial Odometry Positioning By combining IMU data with visual features, the pose of the drone can be calculated. ,in Let be a rotation matrix. As a translation vector, the calculation process satisfies: ， ， in For the drone's velocity vector, For measuring acceleration with an IMU, This represents the gravitational acceleration vector; simultaneously, pose is optimized through visual feature matching, using the FLANN algorithm, with a matching error threshold set to... ( (Euclidean distance between feature points). S3.3 Factor Plot Optimization Constructing factor graphs , where nodes Including UAV pose nodes ( ) and IMU pre-integration node ,side Including visual observation factors IMU pre-integration factor The optimization objective is to minimize the global error: ， in These are visual observations. The covariance matrices of the visual and IMU factors are given, respectively. The Gauss-Newton method is used for iterative solution, with the iteration termination condition being... The final output is a dense indoor map. Optimal pose of the drone .

5. The substation indoor trajectory planning method based on multi-camera heterogeneous UAVs according to claim 1, characterized in that, S4 includes the following steps: S4.1 Reinforcement Learning Path Planning Strategy A deep reinforcement learning (DRL) model is used to define the state space. Operational constraints, including maximum flight speed Minimum turning radius Action space For speed control, Angular velocity control quantity, reward function Designed as follows: ， in Rewards for reaching the target point. The distance between the drone and the obstacle. For indicator functions, Collision penalty is triggered at times. The distance already flown is used to penalize redundant paths; the network adopts a DQN architecture, and the experience replay pool capacity is set to [value missing]. The target network update interval is 1000 steps, and the learning rate is... ; S4.2 Optimized Track Smoothing The initial trajectory output by reinforcement learning ( The input graph optimization model has the following optimization objective: ， The first term is the path length penalty, and the second term is the smoothness penalty. To smooth out the weights, a quadratic programming approach is used to obtain the optimal trajectory. ; S4.3 Dynamic Obstacle Avoidance Control Real-time acquisition of obstacle dynamic information fed back by sensors For the first The position and velocity of each obstacle are used to adjust the trajectory using the artificial potential field method, and the gravitational force acting on the UAV is also considered. Repulsive force The formula is: ( The gravitational coefficient, For the target point, (This refers to the current location of the drone). ( The repulsion coefficient is... For a safe distance, For drones and the first (real-time distance of each obstacle) Overall control ,according to Adjusting the speed and angular velocity of the drone enables dynamic obstacle avoidance.

6. The substation indoor trajectory planning method based on multi-camera heterogeneous UAVs according to claim 1, characterized in that, It also includes the S5 step: multi-drone collaboration and safety control. Through multi-drone map sharing and task allocation, it enables path interlocking and safety redundancy control when multiple drones are conducting collaborative inspections in a substation. Specifically, it includes: S5.1 Multi-machine map sharing Adopting a communication architecture that combines 5G private networks with edge computing, the communication speed... Delay Data transmission uses the TCP / IP protocol and incorporates a redundancy check mechanism. The check formula is: For the first One data byte, For checksum verification, the receiving end verifies... Consistency is required; if inconsistent, retransmission is necessary. S5.2 Task Assignment An improved genetic algorithm is used for task allocation, with the objective function being to minimize the total inspection time. , For the first The mission time for each drone is constrained by the mission volume per drone. To determine the maximum workload for a single drone, the algorithm steps include: Encoding: Integer encoding is used for chromosomes. For the first The drone number assigned to each task; Selection: A roulette wheel selection method is used, with selection probability... , For the first The total time corresponding to each chromosome Population size; Crossover and Mutation: Crossover Probability Probability of mutation ; Iteration termination: When the number of iterations reaches a certain threshold. Or the total time difference between two adjacent generations The process terminates at the appropriate time, and the optimal task allocation scheme is output. S5.3 Path Interlocking and Safety Redundancy Establish a path interlocking mechanism and deploy drones. flight path drones flight path The interlock condition is at any time , ( (to maintain safe distance between multiple machines); if this is violated, path adjustment will be triggered. Safety redundancy control adopts a "master-slave" mechanism. When the master fails (the judgment condition is three consecutive communication interruptions or attitude error), the redundancy is automatically resolved. When this happens, the backup machine immediately takes over the task, switching time. To ensure uninterrupted inspections.

7. The substation indoor trajectory planning method based on multi-camera heterogeneous UAVs according to any one of claims 1 to 6, characterized in that, It also includes model lightweighting and deployment steps, which deploy the depth estimation model to Jetson or RK series edge computing platforms through mixed precision training, model pruning and quantization, achieving a single-frame inference latency of less than 200 milliseconds.

8. A substation indoor trajectory planning system based on multi-camera heterogeneous unmanned aerial vehicles, characterized in that, include: The data governance and database construction module is used to acquire image, depth, infrared and IMU data collected by multi-camera heterogeneous UAVs in the indoor substation, perform preprocessing operations such as denoising, distortion correction, quality assessment and semantic annotation on the data, and build a database with indexing mechanism and version control capabilities. The multi-source heterogeneous sensing fusion module is communicatively connected to the data governance and database construction module. It is used to perform time synchronization and feature alignment processing on the preprocessed data, generate a unified feature representation, and establish a multi-source data fusion model consistent with the preset spatial coordinate system. The deep learning mapping and localization module is communicatively connected to the multi-source heterogeneous perception fusion module. It is used to predict the panoramic depth map based on the unified feature representation, perform pose estimation by combining visual-inertial odometry, and output a high-precision indoor dense 3D map and real-time pose information of the UAV through a factor graph optimization algorithm. The trajectory planning and dynamic obstacle avoidance module is communicatively connected to the deep learning mapping and localization module. It is used to construct a trajectory planning model that integrates reinforcement learning strategies and graph optimization methods based on the dense map and pose information, generate the optimal flight trajectory from the starting point to the target point, and perform online dynamic obstacle avoidance adjustments based on real-time sensor feedback during the UAV's flight.

9. The system according to claim 8, characterized in that, The multi-camera heterogeneous UAV includes at least two types of UAVs equipped with a visible light camera, a depth sensor, an infrared thermal imager, and an inertial measurement unit (IMU), and the data from each sensor has spatiotemporal consistency calibration parameters.

10. The system according to claim 8, characterized in that, The system is deployed on a cloud server or edge computing node and works in collaboration with a multi-camera heterogeneous drone swarm via wireless communication to achieve autonomous navigation and precise positioning for indoor inspection tasks in substations.

Images