An unmanned aerial vehicle pump station inspection control method

Abstract

The application discloses a kind of unmanned aerial vehicle pump station inspection control method, belong to unmanned aerial vehicle indoor inspection technical field.The purpose is to solve the control and positioning problem of unmanned aerial vehicle pump station inspection, including obtaining unmanned aerial vehicle real-time point cloud data, image data and real-time IMU data and carrying out visual feature extraction and position change data calculation, real-time pose estimation is carried out based on visual feature and real-time IMU data, and whether unmanned aerial vehicle crosses floor is judged according to position change data;According to floor loading corresponding static voxel map, and to real-time laser point cloud data processing obtains obstacle voxel, utilizes Kalman filter to predict obstacle dynamic voxel and real-time IMU data, obtains obstacle future position observation value and the future activity point of unmanned aerial vehicle and according to historical flight data to real-time pose is matched, obtains unmanned aerial vehicle control information.The application can avoid positioning drift and misjudgment after unmanned aerial vehicle crosses floor, realizes the high-precision control of pump station inspection.

Description

Technical Field

[0001] This invention relates to a method for controlling the inspection of pump stations using unmanned aerial vehicles (UAVs), belonging to the field of indoor inspection technology for UAVs. Background Technology

[0002] In industrial facilities such as large pumping stations, drones are commonly used for routine inspections. They can monitor the status of important equipment in the pumping station in real time and provide early warnings of emergencies. However, pumping stations are often multi-layered structures, including pump rooms, water tank layers, and pipe layers. When drones inspect pumping stations, they often face the problem of missing GPS signals. Current drone navigation technology relies on simultaneous localization and mapping (SMR) algorithms to achieve compatibility between localization and mapping. Secondly, when existing drones inspect large pumping stations, they need to cross multiple floors. Traditional drone navigation systems mostly use GPS positioning, single visual recognition, or single laser SLAM (Simultaneous Localization and Mapping) algorithms such as LOAM (Light Odometry and Although Mapping) and ORB-SLAM (ORB-Simultaneous Localization and Although Mapping) to achieve PID (Proportional-Integral-Derivative) closed-loop control of the drone's attitude and position in the air. The performance and applicability of various commonly used sensors are analyzed, and some navigation methods based on visual inertial odometry maps are described. However, external interference and cumulative drift during movement are not considered, and no correction mechanism is involved. Therefore, existing drone pumping station inspections are easily affected by various factors, resulting in positioning distortion, and the navigation system does not have a corresponding correction mechanism.

[0003] Secondly, the pump station environment is complex on each floor, and drone inspections often take place in environments with dense steel structures. Sensor signals are frequently interfered with by metal, making it difficult to guarantee positioning accuracy and map continuity. For example, patent CN111486841A discloses a drone navigation and positioning method based on a laser positioning system. This system uses a combination of laser and INS (Inertial Navigation System) for navigation filtering. Although it can obtain the drone's position and attitude in real time, long-term use of the sensors leads to accumulated errors, and laser positioning is susceptible to interference from metal elements, resulting in deviations. This makes it difficult to improve indoor navigation accuracy, obtain effective pose and position data, and has not been optimized for cross-floor scenarios and complex metal environments. Therefore, in current drone pump station inspections, the drone experiences significant cumulative drift and a high misjudgment rate, making it impossible to achieve high-precision navigation control.

[0004] In summary, existing UAV navigation schemes for large pump station inspections, especially those spanning multiple floors, have several defects: (1) Metal interference such as high-intensity point clouds reflected by metal equipment such as turbine units and valves in the pump station leads to positioning distortion, making effective path positioning impossible and further causing global trajectory drift. (2) When UAVs switch floors in the pump station, due to the different scenes and functions of different floors, there is a lack of a dynamic coordinate system switching mechanism, resulting in significant cumulative drift and a high misjudgment rate. (3) Dynamic obstacle avoidance is lagging behind, and the obstacle position cannot be updated in real time to control the UAV to brake and avoid danger. UAVs are prone to collisions with obstacles during inspection, which increases the failure rate and maintenance costs of UAVs. (4) When UAVs are running for a long time, the cumulative errors of various types of data such as lidar, IMU (Inertial Measurement Unit), and camera data are serious, and there is a lack of real-time correction and adjustment mechanisms for flight attitude and flight path, making it impossible to complete the expected inspection tasks. Summary of the Invention

[0005] Objective: To overcome the shortcomings of existing technologies, this invention provides a drone-based pump station inspection control method. During the inspection of large pump stations, especially when the drone traverses floors, feature extraction is performed on real-time laser point cloud data and image data. The method records the edge features of the laser point cloud shape between different floors, the planar features of each floor's scene, and the graphic features of key landmarks to identify the floors the drone traverses, improving the consistency of positioning during floor crossings. Furthermore, corresponding voxel maps are loaded for each floor to obtain obstacle information. A Kalman filter is used to predict and update the dynamic voxels of obstacles and the drone's real-time IMU data to determine their future trajectories, correcting and refining the drone's pump station inspection process. This provides effective reference for the drone to perform path planning, emergency braking, emergency avoidance, and attitude correction. This solves the control and positioning problems of existing drones in pump station inspections, reducing the high misjudgment rate and cumulative error of current drones.

[0006] Technical Solution: First aspect: This invention provides a method for controlling the inspection of pump stations using unmanned aerial vehicles (UAVs), comprising:

[0007] Acquire real-time laser point cloud data, image data, and real-time IMU data of the drone for pump station inspection;

[0008] Feature extraction is performed based on real-time laser point cloud data and image data to obtain the shape and edge features of the real-time laser point cloud, the planar features of each floor scene, and the graphic features of key landmarks; UAV position change data is obtained based on real-time UAV IMU data.

[0009] Real-time pose estimation is performed based on the shape and edge features of real-time laser point cloud, the planar features of scene at each floor, the graphic features of key landmarks, and the real-time IMU data of the UAV to obtain the real-time pose of the UAV.

[0010] Altitude detection is performed on drone position change data to determine whether the drone has crossed floors;

[0011] When crossing floors, feature recognition is performed on the planar features of each floor scene and the graphic features of key landmarks to obtain the floor markers currently being crossed. The real-time pose of the drone is then projected onto the coordinate system of the currently crossed floor to obtain the real-time pose of the drone in the coordinate system of the currently crossed floor.

[0012] Load the pre-built initial map of the pump station based on the floor identifier, and obtain the static voxel map of the current floor spanning;

[0013] Voxelization is performed on real-time laser point cloud data under a static voxel map to obtain real-time point cloud voxel data. The real-time point cloud voxel data is then classified according to predefined obstacle point clouds to obtain obstacle dynamic voxel data.

[0014] By using Kalman filters to predict the dynamic voxel data of obstacles and the real-time IMU data of UAVs, the future position observations of obstacles and the future activity points of UAVs are obtained, respectively.

[0015] Based on the comparison between the future location observations of obstacles and the future activity points of the UAV, and the matching results between historical flight data and the real-time pose of the UAV in the current cross-floor coordinate system, UAV control information is generated.

[0016] Optionally, the method for constructing the initial map of the pumping station includes:

[0017] Pre-acquire 3D point cloud data of all floors of the pump station;

[0018] A graph optimization algorithm was used to filter the 3D point cloud data of each floor to obtain 3D point cloud data with metal reflection points removed.

[0019] Maps are generated using 3D point cloud data with metal reflection points removed, resulting in point cloud maps for each floor. Each floor includes the main unit floor, coupling floor, maintenance floor, and pump floor. A metal strength threshold is defined in the graph optimization algorithm.

[0020] Optionally, real-time pose estimation is performed based on the shape and edge features of the real-time laser point cloud, the planar features of each floor scene, the graphic features of key landmarks, and the real-time IMU data of the UAV to obtain the real-time pose of the UAV, including:

[0021] Based on the shape and edge features of real-time laser point cloud, the planar features of each floor scene, the graphic features of key landmarks, and the real-time IMU data of the unmanned aerial vehicle (UAV), KD-Tree (K-Dimensional Tree, KD Tree algorithm) is used to accelerate matching and obtain the UAV pose and feature acceleration matching results.

[0022] A tightly coupled Kalman filter is used to optimize the UAV pose and feature acceleration matching results to obtain the optimized real-time UAV pose.

[0023] The real-time laser point cloud shape edge features also include point cloud removal based on a preset metal intensity threshold in the graph optimization algorithm, to obtain a real-time laser point cloud with metal interference removed.

[0024] Feature extraction is performed on real-time laser point clouds with metal interference removed to obtain the shape and edge features of laser point clouds with metal reflection points removed in real time.

[0025] Optionally, altitude detection can be performed on the drone's position change data to determine whether the drone has crossed floors, including:

[0026] Based on the comparison between the preset height threshold and the drone's vertical position change data within a set time, it is determined whether the drone has crossed floors. If the drone's vertical position change data within a set time is greater than the preset height threshold, then the drone has crossed floors.

[0027] If the vertical position change data of the drone is less than or equal to the preset height threshold within a set time, the drone has not crossed the floor.

[0028] Optionally, feature recognition is performed on the planar features and key landmark graphic features of each floor to obtain the floor markers currently being traversed, including:

[0029] When the drone crosses floors, key frames containing signs or stair numbers are extracted from the real-time laser point cloud shape edge features, the scene planar features of each floor, and the graphic features of key landmarks.

[0030] Based on a pre-trained pump station floor feature recognition model, feature recognition is performed on keyframes containing signs or stair numbers to determine the floor identifier currently being crossed.

[0031] The pump station floor feature recognition model is built using a YOLOv8 network structure and trained by integrating multimodal data of floor signs, stairs, key equipment, pipeline structures and texture features.

[0032] Optionally, the real-time laser point cloud data is voxelized under a static voxel map to obtain real-time point cloud voxel data. This real-time point cloud voxel data is then classified according to predefined obstacle point clouds to obtain dynamic obstacle voxel data, including:

[0033] Each frame of laser point cloud data after crossing the current floor is divided into a fixed-volume voxel grid to obtain a three-dimensional voxel grid of each frame of point cloud.

[0034] Based on a predefined set of obstacle point clouds, the corresponding 3D voxel grids are labeled to obtain the voxels to which all obstacles belong;

[0035] Based on the continuous frame time sequence, the voxels to which all obstacles belong are dynamically updated to obtain obstacle dynamic voxel data; wherein, the obstacle dynamic voxel data includes the position and velocity dynamics of the obstacles.

[0036] Optionally, based on the comparison between the future location observations of obstacles and the future activity points of the UAV, UAV control information is generated, including:

[0037] The future activity points of the drone are used as path point clouds, and the path point clouds are projected onto a static voxel map to obtain the voxel grid corresponding to the future path of the drone.

[0038] Based on the location observations of obstacles at future times, a static voxel map is found to obtain the static voxel grid associated with the obstacles at future times;

[0039] Update the static voxel grid associated with the obstacle's future time to a dynamic voxel grid;

[0040] Distance calculations are performed on the voxel grid corresponding to the future path of the drone and the updated dynamic voxel grid to obtain the distance between the drone and the obstacle in the future time.

[0041] Based on the distance between the drone and the obstacle in the future time, the set repulsion coefficient and the radius of influence, the repulsion field between the obstacle's future movement path and the drone's flight path is calculated, and the drone control information is output according to the repulsion threshold, which is used to control the drone to brake urgently or output the command to replan the drone's safe path.

[0042] The expression for solving the repulsive field between the future movement path of the obstacle and the flight path of the drone is:

[0043]

[0044] In the formula, The repulsive field between the future movement path of the obstacle and the flight path of the drone. The repulsion coefficient is... This represents the distance between the drone and obstacles in the future.

[0045] Optionally, based on the matching results of historical flight data and the real-time pose of the UAV in the current coordinate system, UAV control information is generated, including:

[0046] Collect laser point cloud data, image data, and IMU data from the historical flight of the UAV;

[0047] Feature extraction is performed based on historical flight laser point cloud data and image data to obtain the shape edge features of historical laser point cloud, the scene planar features of each floor, the graphic features of key landmarks, and to define the historical path for extracting historical flight laser point cloud data, and define the path keyframes.

[0048] Based on the shape and edge features of historical laser point clouds, the planar features of each floor scene, the graphic features of key landmarks, and UAV IMU data, pose estimation is performed to obtain the global historical pose of the UAV and to find the historical pose and visual features associated with the path keyframes.

[0049] Feature extraction is performed on the historical poses and visual features associated with the path keyframes to obtain pose features and visual features, and a bag-of-words model dictionary is constructed.

[0050] The similarity query is performed on the bag-of-words model pre-built from the input of the current path keyframe to obtain the similarity score between the current features and historical features of the path keyframe.

[0051] Based on the comparison results of the similarity scores between the preset similarity threshold and the current and historical features of the path keyframe, loop closure constraint is triggered on the current pose of the UAV path keyframe to obtain the loop-closed corrected current pose of the UAV. The real-time pose of the UAV in the current coordinate system is continuously corrected to obtain the loop-closed corrected real-time pose sequence of the UAV.

[0052] The UAV is controlled to perform attitude correction or reset the UAV IMU inertial measurement unit based on the real-time pose sequence of the loop correction.

[0053] The expression for the closure constraint is:

[0054]

[0055] In the formula, For closure constraints, This is the current pose transformation matrix for the path keyframe. This represents the historical pose transformation matrix of the path keyframes. Let be the relative transformation matrix between the current pose and the historical pose. Let be the covariance matrix.

[0056] Optionally, based on the comparison results of the similarity scores between a preset similarity threshold and the current and historical features of the path keyframes, loop closure constraints are triggered on the current pose of the UAV path keyframes to obtain the loop-closed corrected current pose of the UAV. The real-time pose of the UAV in the current coordinate system is then continuously corrected to obtain the loop-closed corrected real-time pose sequence of the UAV, including:

[0057] When the similarity score between the current feature and the historical feature of the path keyframe is between 90% and 100%, loop closure detection is triggered on the current pose of the UAV path keyframe, and visual features and laser point cloud are extracted from the current frame and the historical frame of the path keyframe, respectively obtaining the laser point cloud shape edge features, scene planar features of each floor, and graphic features of key landmarks in the current frame and the historical frame.

[0058] The laser point cloud shape edge features, scene planar features of each floor, and key landmark graphic features of the current and historical frames are used for KD-Tree (K-Dimensional Tree) accelerated matching to obtain the similarity matching score between the current frame and the historical frames.

[0059] Based on the comparison results of the similarity matching scores of the current frame and historical frames with the preset similarity threshold, multiple candidate frames with higher scores are selected.

[0060] Perform consecutive frame matching verification on multiple candidate frames with high scores to obtain candidate frames that match for three consecutive frames.

[0061] The RANSAC (Random Sample Consensus) algorithm is used to filter the feature pairs of candidate frames that match in three consecutive frames, and the candidate frames that are removed from the mismatched feature pairs are obtained.

[0062] The PNP algorithm (Perspective-n-Point) is used to eliminate candidate frames with mismatched feature pairs and their associated historical frames to calculate the relative pose between the candidate frames and historical frames.

[0063] By adding a loop closure constraint to the relative pose between candidate frames and historical frames, the current pose of the UAV after loop closure correction is obtained.

[0064] Continuous loop-closure constraints are applied to the real-time pose of the UAV in the current coordinate system to obtain a loop-closure corrected real-time pose sequence of the UAV; the loop-closure corrected UAV pose sequence is used to generate the optimal pose trajectory.

[0065] Optionally, the UAV IMU inertial measurement unit is reset based on the loop-closure corrected UAV real-time pose sequence, including:

[0066] The pose drift is calculated by comparing the real-time pose of the UAV with the optimal pose trajectory.

[0067] Based on the comparison between the preset drift threshold and the pose drift amount, it is determined whether to trigger the IMU inertial measurement unit zero-speed correction and reset speed error. When the pose drift amount is between 10cm and 99cm, the IMU inertial measurement unit zero-speed correction and reset speed error are triggered.

[0068] Optionally, the UAV is controlled to perform pose correction based on the loop-corrected real-time pose sequence, including:

[0069] The real-time pose of the UAV with loop closure correction is projected onto the preset desired path to obtain the real-time pose path projection point.

[0070] The lateral error between the projected points of the preset desired path and the real-time pose path is calculated to obtain the lateral error between the real-time pose path and the desired path of the UAV.

[0071] If the lateral error deviation is greater than the set deviation threshold, a lateral error control strategy is adopted to adjust the UAV's attitude angle so that its flight trajectory tends to the desired path.

[0072] The expression for the lateral error control strategy is as follows:

[0073]

[0074] In the formula, This is the yaw angle control value. This is the proportional gain coefficient. For lateral error, The differential gain coefficient, This represents the rate of change of the lateral error. The derivative of the lateral error ex. It is the derivative of time.

[0075] Optionally, the output of the drone emergency braking command based on the repulsion threshold also includes generating a safe speed limit for emergency avoidance to prevent collisions between obstacles and the drone;

[0076] The expression for generating the upper limit of safe speed for emergency avoidance is:

[0077]

[0078] In the formula, The current distance between the drone and the obstacle. This is the maximum deceleration for an emergency stop of the drone. Generate the maximum safe speed for emergency evacuation.

[0079] Beneficial effects: Compared with the prior art, the present invention has the following advantages:

[0080] (1) During the inspection of large pump stations, especially when UAVs cross floors, feature extraction is performed on real-time laser point cloud data and image data. The edge features of the laser point cloud shape between different floors, the planar features of the scene on each floor, and the graphic features of key landmarks are recorded to identify the UAV crossing floors and improve the positioning consistency of floor crossing. According to the corresponding voxel map loaded for different floors, the obstacle information corresponding to different floors is obtained. With the help of Kalman filter, the dynamic voxels of obstacles and the real-time IMU data of UAVs are predicted and updated to determine the future activity trajectory of both, so as to correct and correct the UAV pump station inspection process and provide effective reference for UAVs to perform path planning, emergency braking, emergency avoidance and attitude correction. This solves the problem of high misjudgment rate and cumulative error of UAVs crossing floors in the existing technology and the inability to perform adaptive control according to specific floors, so as to revolutionize the positioning accuracy of cross-floor positioning.

[0081] (2) Introducing a metal noise filtering operation into the graph optimization algorithm can remove metal reflection point clouds based on a preset metal strength threshold during the initial map construction and UAV real-time pose estimation process, thereby avoiding metal interference in the environment and bringing a breakthrough in system robustness.

[0082] (3) By using Kalman filter to predict and update obstacle dynamic voxel data and UAV real-time IMU data, the future trajectories of obstacles and UAVs are updated in real time and mapped onto voxel map to obtain intuitive UAVs and the distance between UAVs and obstacles in the future time. Based on the distance between UAVs and obstacles in the future time, the set repulsion coefficient and the radius of influence, the repulsion field between the future activity path of obstacles and the flight path of UAVs is solved. Based on the repulsion threshold, the indication information of UAV emergency braking or replanning the safe path of UAVs is output. Traditional methods rely on real-time perception data, while this scheme significantly reduces the collision risk by predicting the future position of obstacles and UAVs in advance through Kalman filter, which enables disruptive optimization of dynamic obstacle avoidance and path planning.

[0083] (4) Set a safe speed limit for emergency braking, and combine the dynamic repulsive field, the distance and speed between the UAV and obstacles in the future time and the UAV performance in real time to avoid excessive conservatism or dangerous overspeed caused by fixed repulsive threshold, so as to make the emergency braking safety breakthrough.

[0084] (5) Compared with traditional laser SLAM navigation systems, this invention integrates visual recognition technology and uses a tightly coupled Kalman filter to optimize the UAV pose and feature acceleration matching results to obtain the optimized real-time pose of the UAV, which improves the positioning accuracy and robustness of the UAV when switching in vertical space; multimodal data fusion enables the UAV to maintain high-precision positioning in complex environments such as drastic changes in lighting and dynamic obstacle interference, avoiding positioning interruptions caused by feature mismatch and single sensor failure. Secondly, by integrating multimodal data of signage, stairs, key equipment, pipeline structure and texture features of each floor through the YOLOv8 pump station floor feature recognition model, it is possible to perform feature recognition on key frames of signage, stair numbers, key equipment and obstacle features of each floor, improve the floor recognition accuracy, and enhance the UAV's adaptability to complex environments.

[0085] (6) By designing dynamic coordinate system switching strategies for each floor, the cumulative error of the UAV when passing through the floors can be eliminated. By quantifying the drift amount and comparing it with the preset threshold, the zero-speed correction of the IMU inertial measurement unit can be dynamically triggered, the speed error can be reset in time, and the control accuracy and stability of the UAV inspection can be guaranteed.

[0086] (7) Construct a bag-of-words model pose feature dictionary using historical flight data of UAV inspection and store key frame pose information. Input the current frame pose into the bag-of-words model to match path key frames with high similarity to historical key frames. Then use loop closure detection and verification to confirm loop closure, obtain the corrected optimal pose trajectory and calculate the real-time pose drift. This is used to guide UAV pose correction or reset the IMU inertial measurement unit, which significantly improves the dynamic response speed of the UAV and further provides high-precision, high-real-time pose reference and correction control for UAV pump station inspection.

[0087] (8) Finally, in the process of correction control, based on the expected path corresponding to the UAV inspection task planning and combined with the high-precision pose feedback of loop correction, the trajectory correction of the UAV can be guided globally. The difference between the expected path and the real-time pose of the UAV with loop correction is calculated to obtain the lateral error between the real-time pose of the UAV and the expected path. When the lateral error exceeds the limit, the correction is quickly started. The attitude angle of the UAV is adjusted so that the nose is aligned with the tangent direction of the expected path, so that the real-time attitude trajectory infinitely tends to the expected path and meets the requirements of the inspection task. Attached Figure Description

[0088] Figure 1 This is a flowchart of the unmanned aerial vehicle (UAV) pump station inspection and control method of the present invention;

[0089] Figure 2 This is an example diagram of the point cloud map of each floor of the pump station of the present invention. Detailed Implementation

[0090] To better understand the technical content of the present invention, the technical solution of the present invention will be further introduced and explained below with reference to specific embodiments, but is not limited thereto.

[0091] When drones inspect large pumping stations, especially when traversing multiple floors, they often face positioning distortion due to metal interference. Metal equipment such as turbines and valves within the pumping station easily reflect high-intensity point clouds, and traditional SLAM algorithms often fail to filter out metal noise, leading to point cloud matching errors and pose drift. For example, classic laser SLAM in substation inspections suffers a translation error of 5.87 m due to the lack of handling of metal reflections. Secondly, when drones switch floors within the pumping station, the different scenes and functions of each floor lack a dynamic coordinate system switching mechanism. For instance, traditional laser SLAM in multi-floor underground scenarios suffers from a floor misclassification rate exceeding 10% due to the lack of integration with visual sign recognition. Furthermore, the presence of numerous mobile devices within the pumping station and the resulting lag in dynamic obstacle avoidance—such as the movement of maintenance vehicles and personnel—make existing drone navigation technology ineffective in obstacle avoidance and flight path planning, with obstacle avoidance response delays exceeding 200 ms, easily leading to collisions. Finally, drones carrying multiple sensors are prone to accumulating errors during long-term operation. In particular, the error in IMU zero bias and visual feature matching accumulates over time, leading to global trajectory drift. Furthermore, existing technologies lack corresponding correction methods, and traditional laser SLAM navigation systems do not integrate visual recognition technology, making it impossible to accurately correct the drone's pose based on the scene and key frames. Therefore, drones often face many problems when inspecting pump stations, resulting in high maintenance and management costs for both the drones and the pump stations.

[0092] Example 1

[0093] This embodiment provides a method for controlling the inspection of pump stations using unmanned aerial vehicles (UAVs), such as... Figure 1 The following are included:

[0094] Step 1: Acquire real-time laser point cloud data, image data, and real-time IMU data of the drone for pump station inspection;

[0095] Step 2: Perform feature extraction based on real-time laser point cloud data and image data to obtain the shape and edge features of the real-time laser point cloud, the planar features of each floor scene, and the graphic features of key landmarks; obtain the UAV position change data based on the UAV's real-time IMU data;

[0096] Step 3: Based on the shape and edge features of the real-time laser point cloud, the planar features of each floor scene, the graphic features of key landmarks, and the real-time IMU data of the UAV, perform real-time pose estimation to obtain the real-time pose of the UAV;

[0097] Step 4: Perform altitude detection on the drone's position change data to determine if the drone has crossed floors;

[0098] Step 5: When crossing floors, perform feature recognition on the planar features and key landmark graphic features of each floor scene to obtain the floor markers currently being crossed, and project the real-time pose of the UAV onto the coordinate system of the currently crossed floor to obtain the real-time pose of the UAV in the coordinate system of the currently crossed floor.

[0099] Step 6: Load the pre-built initial map of the pump station based on the floor identifiers to obtain a static voxel map of the current floors spanning the current floors;

[0100] Step 7: Voxelize the real-time laser point cloud data under the static voxel map to obtain real-time point cloud voxel data, and classify the real-time point cloud voxel data according to the predefined obstacle point cloud to obtain obstacle dynamic voxel data.

[0101] Step 8: Predict the future position of the obstacle and the future activity point of the UAV by using the Kalman filter to the dynamic voxel data of the obstacle and the real-time IMU data of the UAV.

[0102] Step 9: Generate UAV control information based on the comparison results of the future location observations of obstacles and the future activity points of the UAV, and the matching results of historical flight data and the real-time pose of the UAV in the current cross-floor coordinate system.

[0103] This embodiment can extract features from real-time laser point cloud data and image data during the inspection of large pump stations, especially when drones cross floors. It records the edge features of laser point cloud shapes between different floors, the planar features of each floor scene, and the graphic features of key landmarks to identify the drone crossing floors, improving the consistency of positioning when crossing floors. It also loads corresponding voxel maps for different floors to obtain obstacle information for each floor. Using a Kalman filter, it predicts and updates the dynamic voxels of obstacles and the real-time IMU data of the drone to determine the future trajectory of both, so as to guide the drone's movement and positioning accuracy during pump station inspection. It controls the drone to perform path planning, emergency braking, emergency avoidance, and attitude correction, solving the problems of high misjudgment rate and cumulative error of drones crossing floors in existing technologies and the inability to perform adaptive control based on specific floors.

[0104] Optionally, the method for constructing the initial map of the pumping station includes:

[0105] Pre-acquire 3D point cloud data of all floors of the pump station;

[0106] A graph optimization algorithm was used to filter the 3D point cloud data of each floor to obtain 3D point cloud data with metal reflection points removed.

[0107] Maps are generated using 3D point cloud data with metal reflection points removed, resulting in point cloud maps for each floor. Each floor includes the main unit floor, coupling floor, maintenance floor, and pump floor. A metal strength threshold is defined in the graph optimization algorithm.

[0108] In this embodiment, the point cloud maps of each floor are as follows: Figure 2As shown in the figure, the pump station has a multi-layered structure, and each layer has different scenes, equipment, and textures, specifically: pump room, water tank layer, pipeline layer, etc. Using the Global Positioning System (GPS) can easily lead to signal loss. Therefore, establishing separate coordinates for each layer can greatly improve the subsequent positioning accuracy and avoid coordinate drift after crossing floors. In addition, this embodiment stores the coordinates of key equipment and point cloud sets of obstacles such as maintenance vehicles.

[0109] Optionally, in step 2, real-time pose estimation is performed based on the shape and edge features of the real-time laser point cloud, the planar features of each floor scene, the graphic features of key landmarks, and the real-time IMU data of the UAV to obtain the real-time pose of the UAV, including:

[0110] Step 2.1: Perform KD-Tree accelerated matching based on the shape and edge features of the real-time laser point cloud, the planar features of each floor scene, the graphic features of key landmarks, and the real-time IMU data of the unmanned aerial vehicle (UAV) to obtain the UAV pose and feature accelerated matching results;

[0111] Step 2.2: Use a tightly coupled Kalman filter to optimize the UAV pose and feature acceleration matching results to obtain the optimized UAV real-time pose;

[0112] The real-time laser point cloud shape edge features also include point cloud removal based on a preset metal intensity threshold in the graph optimization algorithm, to obtain a real-time laser point cloud with metal interference removed.

[0113] Feature extraction is performed on real-time laser point clouds with metal interference removed to obtain the shape and edge features of laser point clouds with metal reflection points removed in real time.

[0114] In this embodiment, under scenarios of rapid movement and changing lighting, the fusion of the tightly coupled method and visual recognition technology to output a 6-axis pose is more capable of sensing changes in the UAV's pose, speed, and angle, and the pose error can be significantly reduced compared to pure vision technology.

[0115] Optionally, step 4 involves performing altitude detection on the drone's position change data to determine whether the drone has crossed floors, including:

[0116] Step 4.1: Based on the comparison between the preset height threshold and the vertical position change data of the drone within a set time, determine whether the drone has crossed the floor. If the vertical position change data of the drone within the set time is greater than the preset height threshold, then the drone has crossed the floor.

[0117] Step 4.2: If the vertical position change data of the drone is less than or equal to the preset height threshold within the set time, the drone has not crossed the floor and continues to detect the height until the position change data is greater than the preset height threshold.

[0118] In this embodiment, the height threshold is preset to 3m, which can be changed according to the actual floor height of the pump station. When the position data of the UAV changes in the vertical direction, the floor coordinate system switching is automatically activated. If the activation of the floor coordinate system switching fails, step 5 is performed to identify the floors crossed.

[0119] Optionally, feature recognition is performed on the planar features and key landmark graphic features of each floor to obtain the floor markers currently being traversed, including:

[0120] Step 5.1: When the drone crosses floors, extract keyframes containing signs or stair numbers from the real-time laser point cloud shape edge features, the scene planar features of each floor, and the graphic features of key landmarks.

[0121] Step 5.2: Based on the pre-trained pump station floor feature recognition model, perform feature recognition on keyframes containing signs or stair numbers to obtain the floor identifier currently being crossed;

[0122] The pump station floor feature recognition model is built using the YOLOv8 (You Only Look Once version 8) network structure, which is an improved version of the YOLO series. It is trained by integrating multimodal data of floor signs, stairs, key equipment, pipeline structures and texture features.

[0123] This embodiment uses the latest version of the YOLO series image recognition model as the base model. The floor signs, stairs, key equipment, pipe structures and texture features in this solution are input into the model to train more accurate floor information, and then the corresponding coordinates are loaded.

[0124] Optionally, in step 7, the real-time laser point cloud data is voxelized under a static voxel map to obtain real-time point cloud voxel data. This real-time point cloud voxel data is then classified according to predefined obstacle point clouds to obtain dynamic obstacle voxel data, including:

[0125] Step 7.1: Divide each frame of laser point cloud data after crossing the current floor into a fixed-volume voxel grid to obtain a three-dimensional voxel grid of each frame of point cloud.

[0126] Step 7.2: Based on the predefined obstacle point cloud set, mark the corresponding 3D voxel grids to obtain the voxels to which all obstacles belong;

[0127] Step 7.3: Update the dynamic information of all voxels to which all obstacles belong according to the continuous frame time sequence to obtain obstacle dynamic voxel data; wherein, the obstacle dynamic voxel data includes the position and velocity dynamics of the obstacles.

[0128] In this embodiment, the dynamic voxel map construction can provide a more refined and intuitive representation of UAV inspection trajectory planning and obstacle path, transforming complex environments into computable data structures and converting multi-sensor data of UAVs into homogenized 3D scenes, solving data alignment problems, and providing refined and global data references and support for emergency braking, emergency avoidance, and prediction of obstacles and future UAV activity trajectories.

[0129] This embodiment uses Kalman filtering to predict the position of obstacles in the next 2 seconds during actual use, and marks the predicted position as a red voxel in real time, providing data reference and support for activating the drone's emergency braking.

[0130] Optionally, in step 9, based on the comparison between the future location observations of the obstacle and the future activity points of the UAV, UAV control information is generated, including:

[0131] Step A1: Use the future activity points of the UAV as the path point cloud, and project the path point cloud onto the static voxel map to obtain the voxel grid corresponding to the future path of the UAV.

[0132] Step B1: Based on the future time position observations of the obstacle, find the static voxel map to obtain the static voxel grid associated with the obstacle in the future time.

[0133] Step C1: Update the static voxel grid associated with the obstacle's future time to a dynamic voxel grid;

[0134] Step D1: Calculate the distance between the drone and the obstacle in the future time by performing distance calculations on the voxel grid corresponding to the future path of the drone and the updated dynamic voxel grid.

[0135] Step E1: Based on the distance between the drone and the obstacle in the future time, the set repulsion coefficient and the radius of influence, solve the repulsion field between the obstacle's future movement path and the drone's flight path, and output drone control information according to the repulsion threshold, which is used to control the drone to brake in an emergency or output a command to replan the drone's safe path.

[0136] The expression for solving the repulsive field between the future movement path of the obstacle and the flight path of the drone is:

[0137]

[0138] In the formula, The repulsive field between the future movement path of the obstacle and the flight path of the drone. The repulsion coefficient is... This represents the distance between the drone and obstacles in the future.

[0139] Optionally, step E1, which outputs the drone emergency braking command based on the repulsion threshold, may also include generating a safe speed limit for emergency avoidance to prevent collisions between obstacles and the drone.

[0140] The expression for generating the upper limit of safe speed for emergency avoidance is:

[0141]

[0142] In the formula, The current distance between the drone and the obstacle. This is the maximum deceleration for an emergency stop of the drone. Generate the maximum safe speed for emergency evacuation.

[0143] This embodiment can perform emergency control and trigger path planning based on obstacle dynamics and corresponding repulsive force fields to avoid collisions between obstacles and drones, reducing drone collision failures. In addition, this embodiment can also calculate and plan the latest path based on the coordinates of key equipment, such as charging pile coordinates, to control the drone to perform autonomous charging path planning during the inspection process, reducing drone power shortage failures.

[0144] Example 2

[0145] This embodiment provides a method for controlling the inspection of pump stations using unmanned aerial vehicles (UAVs), based on the technical concept described in Embodiment 1, including:

[0146] Step 1: Acquire real-time laser point cloud data, image data, and real-time IMU data of the drone for pump station inspection;

[0147] Step 2: Perform feature extraction based on real-time laser point cloud data and image data to obtain the shape and edge features of the real-time laser point cloud, the planar features of each floor scene, and the graphic features of key landmarks; obtain the UAV position change data based on the UAV's real-time IMU data;

[0148] Step 3: Based on the shape and edge features of the real-time laser point cloud, the planar features of each floor scene, the graphic features of key landmarks, and the real-time IMU data of the UAV, perform real-time pose estimation to obtain the real-time pose of the UAV;

[0149] Step 4: Perform altitude detection on the drone's position change data to determine if the drone has crossed floors;

[0150] Step 5: When crossing floors, perform feature recognition on the planar features and key landmark graphic features of each floor scene to obtain the floor markers currently being crossed, and project the real-time pose of the UAV onto the coordinate system of the currently crossed floor to obtain the real-time pose of the UAV in the coordinate system of the currently crossed floor.

[0151] Step 6: Load the pre-built initial map of the pump station based on the floor identifiers to obtain a static voxel map of the current floors spanning the current floors;

[0152] Step 7: Voxelize the real-time laser point cloud data under the static voxel map to obtain real-time point cloud voxel data, and classify the real-time point cloud voxel data according to the predefined obstacle point cloud to obtain obstacle dynamic voxel data.

[0153] Step 8: Predict the future position of the obstacle and the future activity point of the UAV by using the Kalman filter to the dynamic voxel data of the obstacle and the real-time IMU data of the UAV.

[0154] Step 9: Generate UAV control information based on the comparison results of the future location observations of obstacles and the future activity points of the UAV, and the matching results of historical flight data and the real-time pose of the UAV in the current cross-floor coordinate system.

[0155] Optionally, in step 9, based on the matching results of historical flight data and the real-time pose of the UAV in the current coordinate system, UAV control information is generated, including:

[0156] Step A2: Collect laser point cloud data, image data, and IMU data from the UAV's historical flights;

[0157] Step B2: Based on the historical flight laser point cloud data and image data, perform feature extraction to obtain the shape edge features of the historical laser point cloud, the scene planar features of each floor, the graphic features of key landmarks, and define the historical path for extracting the historical flight laser point cloud data, and define the path keyframes;

[0158] Step C2: Based on the shape and edge features of historical laser point cloud, the planar features of each floor scene, the graphic features of key landmarks, and UAV IMU data, pose estimation is performed to obtain the global historical pose of the UAV and to find the historical pose and visual features associated with the path keyframes.

[0159] Step D2: Extract features from the historical poses and visual features associated with the path keyframes to obtain pose features and visual features, and construct a bag-of-words model dictionary;

[0160] Step E2: Perform a similarity query on the pre-built bag-of-words model of the input of the current path keyframe to obtain the similarity score between the current features and historical features of the path keyframe;

[0161] Step F2: Based on the comparison results of the similarity scores between the preset similarity threshold and the current and historical features of the path keyframe, trigger the loop closure constraint on the current pose of the UAV path keyframe to obtain the loop closure corrected current pose of the UAV, and continuously correct the real-time pose of the UAV in the current coordinate system to obtain the loop closure corrected real-time pose sequence of the UAV.

[0162] Step G2: Control the UAV to perform pose correction or reset the UAV IMU inertial measurement unit based on the loop-corrected UAV real-time pose sequence.

[0163] The expression for the closure constraint is:

[0164]

[0165] In the formula, For closure constraints, This is the current pose transformation matrix for the path keyframe. This represents the historical pose transformation matrix of the path keyframes. Let be the relative transformation matrix between the current pose and the historical pose. Let be the covariance matrix.

[0166] Optionally, in step F2, based on the comparison result of the similarity scores between the current features and historical features of the path keyframes according to the preset similarity threshold, loop closure constraints are triggered on the current pose of the UAV path keyframes to obtain the loop-closure corrected current pose of the UAV. The real-time pose of the UAV in the current coordinate system is then continuously corrected to obtain the loop-closure corrected real-time pose sequence of the UAV, including:

[0167] Step F2-1: When the similarity score between the current feature and the historical feature of the path keyframe is greater than 90%, loop closure detection is triggered on the current pose of the UAV path keyframe, and visual features and laser point cloud are extracted from the current frame and the historical frame of the path keyframe, respectively obtaining the laser point cloud shape edge features, scene planar features of each floor, and graphic features of key landmarks in the current frame and the historical frame.

[0168] Step F2-2: Perform KD-Tree accelerated matching on the laser point cloud shape edge features, scene planar features of each floor, and graphic features of key landmarks in the current and historical frames to obtain the similarity matching score between the current and historical frames;

[0169] Step F2-3: Based on the comparison results of the similarity matching scores of the current frame and historical frames with the preset similarity threshold, select multiple candidate frames with higher scores;

[0170] Step F2-4: Perform consecutive frame matching verification on multiple candidate frames with high scores to obtain candidate frames that match for three consecutive frames.

[0171] Step F2-5: Use the RANSAC algorithm to filter the feature pairs of candidate frames that match in three consecutive frames, and obtain candidate frames that have eliminated mismatched feature pairs.

[0172] Step F2-6: Use the PNP algorithm to remove candidate frames with mismatched feature pairs and their associated historical frames to calculate the relative pose, and obtain the relative pose between the candidate frames and historical frames;

[0173] Step F2-7: Add closure constraints to the relative pose between the candidate frame and the historical frame to obtain the current pose of the UAV after closure correction;

[0174] Repeat steps F2-1 to F2-7 to continuously apply loop closure constraints to the real-time pose of the UAV in the current coordinate system to obtain the loop closure corrected UAV real-time pose sequence; wherein, the loop closure corrected UAV pose sequence is used to generate the optimal pose trajectory.

[0175] This embodiment uses a dictionary model and loop closure detection to detect and correct the UAV's real-time attitude, making the UAV's attitude infinitely close to the attitude of historical keyframes, and optimizing the trajectory to eliminate accumulated errors, so that the trajectory of each inspection meets the expected requirements and avoids flight trajectory drift.

[0176] Optionally, step G2 involves resetting the UAV IMU (Inertial Measurement Unit) based on the loop-closed corrected UAV real-time pose sequence, including:

[0177] Step G2-1: Calculate the pose drift between the real-time pose and the optimal pose trajectory of the UAV to obtain the pose drift amount;

[0178] Step G2-2: Based on the comparison result between the preset drift threshold and the pose drift amount, determine whether to trigger the IMU inertial measurement unit zero-speed correction and reset speed error. When the pose drift amount is greater than 10 cm, trigger the IMU inertial measurement unit zero-speed correction and reset speed error.

[0179] Optionally, step G2 involves controlling the UAV to perform pose correction based on the loop-corrected real-time UAV pose sequence, including:

[0180] Step G2-A: Project the loop-corrected UAV real-time pose onto the preset desired path to obtain the real-time pose path projection points;

[0181] Step G2-B: Calculate the lateral error between the preset desired path and the real-time pose path projection points to obtain the lateral error between the UAV's real-time pose path and the desired path.

[0182] Step G2-C: If the lateral error deviation is greater than the set deviation threshold, the lateral error control strategy is adopted to adjust the UAV attitude angle so that its flight trajectory tends to the desired path.

[0183] The expression for the lateral error control strategy is as follows:

[0184]

[0185] In the formula, This is the yaw angle control value. This is the proportional gain coefficient. For lateral error, The differential gain coefficient, This represents the rate of change of the lateral error. The derivative of the lateral error ex. It is the derivative of time.

[0186] This embodiment takes into account the cumulative errors from long-term operation of multiple sensors, and designs the closed-loop correction of sensor data and the dynamic path adjustment of the UAV as parallel processes, achieving a synergistic effect of error suppression and control accuracy. It also gives the invention a good fault tolerance mechanism; when any sensor fails, the flight path can be adjusted in real time through lateral error control. When there are large errors during flight that cannot be satisfied by lateral error control, the sensor deviation is corrected at the source to eliminate subsequent error accumulation. Therefore, the UAV always possesses a relatively accurate, real-time, and effective correction capability during flight, ultimately enabling the UAV to navigate large and complex pumping stations and other indoor locations.

[0187] In summary, this invention extracts features from real-time laser point cloud data and image data during the inspection of large pump stations, especially when drones traverse floors. It records the edge features of the laser point cloud shape between different floors, the planar features of the scene on each floor, and the graphic features of key landmarks to identify drones traversing floors, improving the consistency of positioning during floor crossings. Furthermore, it obtains obstacle information for each floor by loading corresponding voxel maps, and uses a Kalman filter to predict and update the dynamic voxels of obstacles and the drone's real-time IMU data to determine their future trajectories. This corrects and refines the drone's pump station inspection process, providing effective reference for path planning, emergency braking, emergency avoidance, and attitude correction. This invention solves the problems of high misjudgment rates and cumulative errors in existing technologies when drones traverse floors, and the inability to adaptively control based on specific floors, resulting in a revolutionary improvement in cross-floor positioning accuracy.

[0188] This invention introduces a metal noise filtering operation into the graph optimization algorithm. It removes metal reflection point clouds based on a pre-set metal strength threshold during initial map construction and real-time UAV pose estimation, avoiding metal interference with the environment and significantly improving system robustness. By using a Kalman filter to predict and update obstacle dynamic voxel data and real-time UAV IMU data, the future trajectories of obstacles and the UAV are updated in real time and mapped onto a voxel map to obtain an intuitive view of the UAV and the distance between the UAV and obstacles in the future. Based on the distance between the UAV and obstacles in the future, a set repulsion coefficient, and an influence radius, the repulsion field between the obstacle's future movement path and the UAV's flight path is calculated. Based on the repulsion threshold, instructions for emergency braking or replanning a safe path for the UAV are output. Traditional methods rely on real-time sensing data, while this scheme significantly reduces collision risk by predicting the future positions of obstacles and the UAV in advance using Kalman filtering, enabling revolutionary optimization of dynamic obstacle avoidance and path planning.

[0189] This invention sets a safe speed limit for emergency braking, and combines a dynamically changing repulsive field with the distance and speed of the drone and obstacles in the future time and the drone's performance in real time calculations. This avoids overly conservative or dangerous speeding caused by a fixed repulsive threshold, thus enabling a breakthrough in emergency braking safety.

[0190] Compared to traditional laser SLAM navigation systems, this invention integrates visual recognition technology and utilizes a tightly coupled Kalman filter to optimize the UAV's pose and feature acceleration matching results, resulting in an optimized real-time pose. This improves the UAV's positioning accuracy and robustness during vertical space transitions. Multimodal data fusion enables the UAV to maintain high-precision positioning even in complex environments such as those with drastic lighting changes and dynamic obstacle interference, avoiding positioning interruptions caused by feature mismatches and single sensor failures. Furthermore, by fusing multimodal data from floor signs, stairs, key equipment, pipeline structures, and texture features using the YOLOv8 pump station floor feature recognition model, it can perform feature recognition on keyframes of floor signs, stair numbers, key equipment, and obstacle features, improving floor recognition accuracy and enhancing the UAV's adaptability to complex environments.

[0191] This invention eliminates the accumulated error when the UAV passes through floors by designing a dynamic coordinate system switching strategy for each floor. By quantifying the drift amount and comparing it with a preset threshold, the zero-speed correction of the IMU inertial measurement unit can be dynamically triggered to reset the speed error in time, ensuring the control accuracy and stability of the UAV during the inspection process.

[0192] This invention utilizes historical flight data from UAV inspections to construct a bag-of-words model pose feature dictionary, storing keyframe pose information. The current frame pose is input into the bag-of-words model to match path keyframes with high similarity to historical keyframes. Loop closure detection and verification are then used to confirm loop closures, obtaining the corrected optimal pose trajectory and calculating the real-time pose drift. This drift is then used to guide subsequent UAV pose correction or IMU (Inertial Measurement Unit) reset, significantly improving the UAV's dynamic response speed and providing high-precision, high-real-time pose reference and correction control for UAV pump station inspections.

[0193] Finally, in the process of correction control, this invention, based on the expected path planned for the UAV inspection task and combined with the high-precision pose feedback of loop correction, can guide the global trajectory correction of the UAV. It calculates the difference between the expected path and the real-time pose of the UAV with loop correction, obtains the lateral error between the real-time pose of the UAV and the expected path, and quickly starts correction when the lateral error exceeds the limit. It adjusts the attitude angle of the UAV so that the nose is aligned with the tangent direction of the expected path, so that the real-time attitude trajectory infinitely approaches the expected path, thus meeting the requirements of the inspection task.

[0194] Embodiments of this application may be provided as methods, systems, or computer program products. Therefore, this application may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0195] Embodiments of this application may be provided as methods, systems, or computer program products. Therefore, this application may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0196] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0197] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0198] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0199] The above description is only a preferred embodiment of the present invention. Without departing from the technical principle of the present invention, several improvements and modifications can be made, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for unmanned inspection and control of a pump station, characterized in that, The application relates to a method for unmanned aerial vehicle (UAV) pump station inspection, and belongs to the field of intelligent inspection. Real-time laser point cloud data, image data and real-time IMU data of the UAV pump station inspection are acquired; Real-time laser point cloud shape edge features, each floor scene plane features and key landmark graphic features are obtained by performing feature extraction on the real-time laser point cloud data and the image data; and UAV position change data is obtained according to the real-time IMU data of the UAV; Real-time pose estimation is performed on the real-time laser point cloud shape edge features, the each floor scene plane features, the key landmark graphic features and the real-time IMU data of the UAV, so as to obtain real-time pose of the UAV; Height detection is performed on the UAV position change data, so as to determine whether the UAV has crossed a floor; When the UAV crosses a floor, feature recognition is performed on the each floor scene plane features and the key landmark graphic features, so as to obtain a floor identifier of the current crossing floor; and the real-time pose of the UAV is projected into a coordinate system of the current crossing floor, so as to obtain real-time pose of the UAV in the coordinate system of the current crossing floor; A static voxel map of the current crossing floor is obtained by loading a pre-constructed initial map of the pump station according to the floor identifier; Real-time point cloud voxel data is obtained by performing voxelization processing on the real-time laser point cloud data in the static voxel map; and the real-time point cloud voxel data is classified according to pre-defined obstacle point clouds, so as to obtain obstacle dynamic voxel data; The obstacle dynamic voxel data and the real-time IMU data of the UAV are predicted by a Kalman filter, so as to obtain an obstacle future position observation value and a future activity point of the UAV respectively; According to a comparison result of the obstacle future position observation value and the future activity point of the UAV and a matching result of historical flight data and the real-time pose of the UAV in the coordinate system of the current crossing floor, UAV control information is generated; According to the matching result of the historical flight data and the real-time pose of the UAV in the coordinate system, the UAV control information is generated, including: Laser point cloud data, image data and IMU data of historical flight of the UAV are collected; Historical laser point cloud shape edge features, each floor scene plane features and key landmark graphic features are obtained by performing feature extraction on the historical laser point cloud data and the image data; and a historical flight path is defined according to the historical laser point cloud data; and a path key frame is defined; Global historical pose of the UAV is obtained by performing pose estimation on the historical laser point cloud shape edge features, the each floor scene plane features, the key landmark graphic features and the IMU data of the UAV; and historical pose and visual features associated with the path key frame are found out; Pose features and visual features are obtained by performing feature extraction on the historical pose and the visual features associated with the path key frame; and a bag-of-words model dictionary is constructed; Similarity query is performed on an input path key frame of the pre-constructed bag-of-words model, so as to obtain a similarity score of current features and historical features of the path key frame. When the similarity score of the current feature and the historical feature of the path key frame is between 90% and 100%, loop detection of the current pose of the unmanned aerial vehicle path key frame is triggered, and the current frame and the historical frame of the path key frame are subjected to visual feature and laser point cloud extraction, so as to obtain the laser point cloud shape edge feature, the scene plane feature of each floor, and the key landmark graphic feature of the current frame and the historical frame respectively; The laser point cloud shape edge feature, the scene plane feature of each floor, and the key landmark graphic feature of the current frame and the historical frame are subjected to KD-Tree accelerated matching, so as to obtain the similarity matching score of the current frame and the historical frame; According to the comparison result of the preset similarity threshold and the similarity matching score of the current frame and the historical frame, a plurality of candidate frames with a higher score than others are selected; The plurality of candidate frames with a higher score than others are subjected to continuous frame matching verification respectively, so as to obtain a candidate frame matched by three continuous frames; The RANSAC algorithm is used to screen feature pairs of the candidate frame matched by three continuous frames, so as to obtain a candidate frame with false matching feature pairs removed; The PNP algorithm is used to calculate the relative pose of the candidate frame with false matching feature pairs removed and the associated historical frame, so as to obtain the relative pose between the candidate frame and the historical frame; Loop constraint is added to the relative pose between the candidate frame and the historical frame, so as to obtain the current pose of the unmanned aerial vehicle after loop correction; The real-time pose of the unmanned aerial vehicle in the current coordinate system is subjected to continuous loop constraint, so as to obtain the real-time pose sequence of the unmanned aerial vehicle after loop correction; wherein, the real-time pose sequence of the unmanned aerial vehicle after loop correction is used to generate an optimal pose trajectory; According to the real-time pose sequence of the unmanned aerial vehicle after loop correction, the unmanned aerial vehicle is controlled to perform pose rectification or reset the IMU (inertial measurement unit) of the unmanned aerial vehicle; The expression of the loop constraint is: ； wherein is the loop constraint, is the current pose transformation matrix of the path keyframe, is the historical pose transformation matrix of the path keyframe, is the relative transformation matrix between the current and historical poses, is the covariance matrix.

2. The unmanned pump station inspection control method of claim 1, wherein, The method for constructing the initial map of the pump station comprises: Three-dimensional point cloud data of each floor of the pump station is acquired in advance; A graph optimization algorithm is used to filter the three-dimensional point cloud data of each floor, so as to obtain three-dimensional point cloud data with metal reflection points removed; A map is generated by using the three-dimensional point cloud data with metal reflection points removed, so as to obtain a point cloud map of each floor; wherein, each floor comprises a main machine layer, a coupling layer, a maintenance layer, and a water pump layer, and a metal intensity threshold is defined in the graph optimization algorithm.

3. The unmanned pump station inspection control method of claim 1, wherein, Real-time pose estimation is performed according to real-time laser point cloud shape edge features, scene plane features of each floor, key landmark graphic features, and real-time IMU data of the unmanned aerial vehicle, so as to obtain the real-time pose of the unmanned aerial vehicle, comprising: KD-Tree accelerated matching is performed according to real-time laser point cloud shape edge features, scene plane features of each floor, key landmark graphic features, and real-time IMU data of the unmanned aerial vehicle, so as to obtain the pose and feature accelerated matching result of the unmanned aerial vehicle; A tight coupling Kalman filter is used to optimize the pose and feature accelerated matching result of the unmanned aerial vehicle, so as to obtain the optimized real-time pose of the unmanned aerial vehicle; The real-time laser point cloud shape edge features further comprise removing metal interference from real-time laser point cloud based on the metal intensity threshold preset in the graph optimization algorithm. Feature extraction is performed based on the real-time laser point cloud with metal reflection points removed, so as to obtain real-time laser point cloud shape edge features with metal reflection points removed.

4. The unmanned pump station inspection control method of claim 1, wherein, The height detection is performed on the unmanned aerial vehicle position change data to determine whether the unmanned aerial vehicle crosses the floor, including: According to the comparison result of the preset height threshold and the position change data of the unmanned aerial vehicle in the vertical direction within the set time, it is determined whether the unmanned aerial vehicle crosses the floor. When the position change data of the unmanned aerial vehicle in the vertical direction within the set time is greater than the preset height threshold, the unmanned aerial vehicle crosses the floor. When the position change data of the unmanned aerial vehicle in the vertical direction within the set time is less than or equal to the preset height threshold, the unmanned aerial vehicle does not cross the floor.

5. The unmanned pump station inspection control method of claim 1, wherein, The feature recognition is performed on the scene plane features and key landmark graphic features of each floor to obtain the current crossed floor identification, including: When the unmanned aerial vehicle crosses the floor, the key frame containing the sign or stair number is extracted from the real-time laser point cloud shape edge feature, the scene plane feature of each floor, and the key landmark graphic feature. The key frame containing the sign or stair number is subjected to feature recognition based on a pre-trained pump station floor feature recognition model to obtain the current crossed floor identification. The pump station floor feature recognition model is built using a YOLOv8 network structure and is trained by fusing multi-modal data of each floor sign, stair, key equipment, pipeline structure, and texture feature.

6. The unmanned pump station inspection control method of claim 1, wherein, The real-time laser point cloud data is subjected to voxelization processing under the static voxel map to obtain real-time point cloud voxel data and classify the real-time point cloud voxel data according to pre-defined obstacle point clouds to obtain obstacle dynamic voxel data, including: Each frame of laser point cloud data after crossing the floor is segmented into a fixed volume of voxel grid to obtain a three-dimensional voxel grid of each frame of point cloud. Based on a pre-defined obstacle point cloud set, the corresponding three-dimensional voxel grid is marked to obtain the voxel belonging to all obstacles. The dynamic information of the voxel belonging to all obstacles is updated according to the time sequence of consecutive frames to obtain obstacle dynamic voxel data. The obstacle dynamic voxel data includes the position and speed dynamics of the obstacle.

7. The unmanned pump station inspection control method of claim 6, wherein, Based on the comparison result of the future position observation value of the obstacle and the future activity point of the unmanned aerial vehicle, the unmanned aerial vehicle control information is generated, including: The future activity point of the unmanned aerial vehicle is taken as a path point cloud, and the path point cloud is projected into the static voxel map to obtain the voxel grid corresponding to the future path of the unmanned aerial vehicle. The static voxel map is searched according to the position observation value of the obstacle at the future time to obtain the static voxel grid associated with the obstacle at the future time. The static voxel grid associated with the obstacle at the future time is updated to a dynamic voxel grid. The distance between the unmanned aerial vehicle and the obstacle within the future time is calculated based on the distance between the unmanned aerial vehicle and the obstacle within the future time, the set repulsive force coefficient, and the influence radius. The repulsive force field between the future activity path of the obstacle and the flight path of the unmanned aerial vehicle is solved based on the distance between the unmanned aerial vehicle and the obstacle within the future time, the set repulsive force coefficient, and the influence radius, and the unmanned aerial vehicle control information is output according to the repulsive force threshold to control the unmanned aerial vehicle to brake or output a re-planned safe path instruction of the unmanned aerial vehicle. The expression for solving the repulsive force field between the future activity path of the obstacle and the flight path of the unmanned aerial vehicle is: ； wherein, is a repulsive force field between the future activity path of the obstacle and the flight path of the UAV, is a repulsive force coefficient, is the distance between the UAV and the obstacle in the future time, is the distance between the current UAV and the obstacle.

8. The unmanned pump station inspection control method of claim 1, wherein, The unmanned aerial vehicle IMU (inertial measurement unit) is reset according to the loop-corrected real-time pose sequence of the unmanned aerial vehicle, including: The pose drift amount is calculated for the real-time pose of the unmanned aerial vehicle and the optimal pose trajectory, and the pose drift amount is obtained; According to the comparison result of the preset drift threshold and the pose drift amount, it is judged whether the IMU inertial measurement unit zero speed correction and the reset speed error are triggered, and when the pose drift amount is between 10 cm and 99 cm, the IMU inertial measurement unit zero speed correction and the reset speed error are triggered.

9. The unmanned pump station inspection control method of claim 1, wherein, The real-time pose sequence of the unmanned aerial vehicle is controlled according to the loop correction, including: The real-time pose of the unmanned aerial vehicle is projected onto the preset expected path to obtain a real-time pose path projection point; The lateral error is calculated for the preset expected path and the real-time pose path projection point, and the lateral error between the real-time pose path of the unmanned aerial vehicle and the expected path is obtained; If the lateral error deviation is greater than the set deviation threshold, a lateral error control strategy is adopted to adjust the attitude angle of the unmanned aerial vehicle, so that the flight trajectory tends to the expected path; The expression of the lateral error control strategy is: ； wherein is a yaw angle control amount, is a proportional gain coefficient, is a lateral error, is a differential gain coefficient, is a lateral error change rate, is a differential of the lateral error ex, is a differential of time.

10. The unmanned pump station inspection control method of claim 7, wherein, According to the repulsive force threshold, the emergency braking instruction of the unmanned aerial vehicle is output, which further includes generating a safety speed upper limit value for emergency avoidance, to avoid collision between the obstacle and the unmanned aerial vehicle; The expression of the safety speed upper limit value for emergency avoidance is: ； In the formula, is the distance between the UAV and the obstacle, is the maximum deceleration of the emergency stop of the UAV, generates the upper limit value of the safety speed of the emergency escape.

Images