Wall-climbing robot positioning and path planning method based on unmanned aerial vehicle cooperation

Abstract

The invention discloses a wall-climbing robot positioning and path planning method based on unmanned aerial vehicle cooperation, and the method comprises the steps: constructing and updating a three-dimensional map based on the point cloud and image data of a laser radar and a camera; based on the three-dimensional map, the two-dimensional code of the wall-climbing robot is identified by using the unmanned aerial vehicle, and the accurate positioning information of the wall-climbing robot is obtained by combining self-positioning and cross validation and fusion correction of the ground control unit; updating a map by synchronizing sensing data, and optimizing a climbing track based on the obtained positioning information and a global planning path of the ground control unit; for positioning information and environment information of the wall-climbing robot, the unmanned aerial vehicle adopts a trajectory prediction control algorithm to adjust the attitude, and meanwhile, sudden obstacles are avoided in real time through path planning. According to the invention, through laser vision fusion SLAM and unmanned aerial vehicle collaborative operation, the environment perception capability is enhanced, and through combination of a dynamic map and visual word bag optimization, the functions of operation map construction, accurate positioning, path planning, autonomous navigation obstacle avoidance and the like of the wall-climbing robot are realized.

Description

Technical Field

[0001] The present invention relates to the technical field of robot positioning and path planning, and in particular to a positioning and path planning method for a wall-climbing robot based on unmanned aerial vehicle collaboration. Background Art

[0002] In practical applications, wall-climbing robots may encounter obstacles such as welds and pits. If the system's environmental perception is insufficient or its response is slow, the robot can easily fall off the wall. Therefore, improving the system's environmental perception and response speed is key to overcoming the robot's operational limitations in complex environments.

[0003] The current technology system primarily relies on sensors onboard the robot, such as lidar and cameras, for environmental perception and positioning. However, these sensors have limitations due to the robot's structural characteristics: in close-range perception mode, the lidar's horizontal scan is easily disturbed by the uneven features of the wall, resulting in point cloud distortion, while the vertical detection range is limited, making it difficult to construct a complete three-dimensional representation of the operating scene. Visual sensors are prone to feature point mismatch on low-texture surfaces, such as the polished surface of storage tanks, and are particularly prone to positioning drift or path planning failure in dynamic or poorly lit scenarios. When the robot enters a structural blind spot, the sensor's limited field of view will directly lead to the loss of local maps, causing the SLAM system to crash.

[0004] Therefore, there is an urgent need to design a positioning and path planning method for UAV-based wall-climbing robots that integrates efficient collaborative positioning and multimodal perception fusion. This method can enhance the robot's environmental perception in complex three-dimensional environments, enabling rapid and accurate positioning, and ultimately enabling autonomous navigation, obstacle avoidance, and smooth operational execution. This method is suitable for painting, rust removal, and cleaning operations on large, complex structures such as ship exteriors, storage tank interiors, and wind turbine towers. Summary of the Invention

[0005] The purpose of the present invention is to overcome the shortcomings of the existing technology. To achieve the above purpose, a wall-climbing robot positioning and path planning method based on drone cooperation is adopted to solve the problems raised in the above background technology.

[0006] A method for positioning and path planning of a wall-climbing robot based on collaboration with unmanned aerial vehicles (UAVs) comprises the following steps:

[0007] Step S1: Based on the point cloud and image data collected by the lidar and camera, a three-dimensional map in the global coordinate system is constructed and updated through segmentation, registration, pose graph optimization, and visual loop closure detection;

[0008] Step S2: Based on the constructed three-dimensional map in the global coordinate system, the UAV is used to identify the QR code of the wall-climbing robot and combine its own positioning with the cross-verification and fusion correction of the ground control unit to obtain the precise positioning information of the wall-climbing robot in the global three-dimensional map;

[0009] Step S3: Synchronously integrating the perception data of the UAV and the wall-climbing robot to update the map; the wall-climbing robot optimizes the climbing trajectory using a dynamic path planning algorithm based on the obtained positioning information and the global planning path of the ground control unit;

[0010] Step S4: Based on the positioning information and environmental information of the wall-climbing robot, the UAV uses a trajectory prediction control algorithm to adjust its posture to maintain a preset distance tracking, and at the same time avoids sudden obstacles in real time through path planning.

[0011] As a further solution of the present invention: the specific steps in step S1 include:

[0012] Step S11: Collecting lidar point cloud and image data respectively through the lidar and camera carried by the drone;

[0013] Step S12: perform feature extraction, matching, and local motion estimation on the preprocessed data, segment the point cloud into local sub-blocks, and align adjacent sub-blocks to the local coordinate system using a point cloud registration algorithm and a feature matching algorithm to form a preliminary coherent local map;

[0014] Step S13: Use the sub-blocks as nodes and the transformation relationships between adjacent sub-blocks as edges to construct a pose graph, identify overlapping areas through the visual word bag, and add closed-loop constraints;

[0015] Step S14: Using the optimized pose graph, coordinate system alignment is performed to transform all sub-blocks into a unified global coordinate system;

[0016] Step S15: Merge all sub-block point clouds in the global coordinate system into a complete scene point cloud, convert the point cloud into a spatial index structure according to the resolution, and obtain and update the global map.

[0017] As a further solution of the present invention, the process of dividing the sub-blocks and forming the local map in step S1 is as follows:

[0018] The first step is to enhance data feature extraction capabilities through geometric feature analysis methods, plane fitting algorithms, or semantic segmentation networks to obtain the geometric characteristics of the lidar point cloud and texture information in the image.

[0019] The second step is to use the point cloud registration algorithm and spatial index model to complete the feature matching of the lidar scanning data, extract the image feature descriptor, use the fast nearest neighbor approximation search method to achieve feature matching, and use the false match filtering algorithm to filter out false matches;

[0020] The third step is to obtain the depth information of the pixels in the image and complete the pose estimation through the local motion estimation algorithm;

[0021] Step 4: Combining the geometric characteristics of the point cloud with the texture information of the image, we achieve refined segmentation that coordinates geometric structure and visual features. At the same time, we introduce a dynamic segmentation strategy to adjust the local segmentation based on the real-time motion trajectory of the drone to ensure map continuity.

[0022] Step 5: Jointly optimize the multimodal features extracted from the current frame with the historical sub-blocks. By minimizing the reprojection error of the laser point cloud features and the geometric consistency error of the visual features, a fusion is generated to generate a local sub-map containing geometric structure, texture information, and semantic labels.

[0023] Step 6: After each sub-map passes the spatiotemporal consistency test, its central position, feature distribution, and boundary range are recorded to provide input for subsequent pose graph construction and global optimization.

[0024] As a further solution of the present invention, the process of constructing the pose graph and the visual word bag closed loop detection in step S1 is as follows:

[0025] The first step is to initialize the center position and posture of each submap as a pose graph node, and add the relative pose transformation of adjacent submaps obtained by registration algorithm and visual feature matching as edge constraints into the graph;

[0026] In the second step, the feature descriptors extracted from the submaps are similarly searched using the bag-of-visual-words model, and historical submaps with overlapping areas are selected as closed-loop candidates. When a closed-loop candidate is detected, a geometric consistency verification framework is used to remove mismatched feature points using RANSAC and calculate the relative pose transformation of the candidate closed loop. If the error is lower than a preset threshold and the spatial translation is within the inertial measurement error range, the closed-loop constraint is added as a new edge to the pose graph.

[0027] The third step is to use the graph optimization algorithm to iteratively optimize the global pose graph.

[0028] As a further solution of the present invention: in step S1, the visual word bag method is used to perform visual closed-loop detection, and the K-means clustering method is used to train a binary offline word bag containing various image features. The process of training the visual word bag is: clustering each image frame according to its features and descriptors, and after clustering, numbering each image frame to generate a visual word bag; in the closed-loop or key frame detection process, the current frame image information is converted into a word bag vector that can be recognized by the word bag, and similar image frames are searched in the word bag through similarity measurement. If they exist, it is determined that a closed-loop or key frame match is detected, and then the relative motion between the two frames is calculated to optimize the current posture.

[0029] As a further solution of the present invention, the process of transforming all sub-blocks into a unified global coordinate system in step S1 is as follows:

[0030] The first step is to traverse all submaps and extract their central pose parameters based on the optimized submap node pose. The rigid transformation matrix of each submap from the local coordinate system to the global coordinate system is calculated through coordinate transformation matrix operations.

[0031] In the second step, the corresponding transformation matrix is ​​applied to the point cloud data of each submap to uniformly convert its coordinates, normal vectors, and semantic labels to the global coordinate system. At the same time, the point cloud registration residuals of adjacent submaps in the overlapping area are detected. If the mean residual exceeds the preset threshold, the global registration fine-tuning is triggered.

[0032] In the third step, multi-resolution point cloud registration is used to perform non-rigid transformation optimization on the sub-map point cloud in the conflict area, and its transformation matrix is ​​iteratively updated until the residual converges.

[0033] As a further solution of the present invention: software and hardware collaboration is achieved through the ROS framework, an adaptive self-organizing network communication network is adopted at the communication level, and signal relay units are deployed in visual blind spots, which prioritize the transmission of positioning data and compress point cloud images to ensure that key instructions can be delivered with low latency;

[0034] When the wall-climbing robot encounters signal interruption or positioning loss, the drone switches to pure visual tracking mode, providing temporary coordinate references through the target detection algorithm, while the wall-climbing robot enables the local cache path to continue operating until communication is restored.

[0035] As a further solution of the present invention: the specific steps in step S2 include:

[0036] Step S21: When starting up, the ground control unit first loads a pre-stored 3D map, unifies the sensor data of the UAV and the wall-climbing robot into the global coordinate system of the map through a coordinate conversion protocol, and uses a sensor calibration tool to ensure that the external parameters of each sensor are aligned;

[0037] Step S22: After the wall-climbing robot is started, its onboard laser radar and camera begin to scan the surrounding environment, extract wall feature points, match the real-time features with the key frames in the 3D map through the multi-sensor fusion SLAM algorithm, calculate the initial pose in combination with graph optimization technology, and use IMU data to compensate for motion distortion to complete the initial positioning in the map;

[0038] Step S23: The UAV is lifted off from the take-off point to a predetermined height. The onboard camera scans the operating area using a preset visual recognition module and selects targets within the visual range based on the expected position range of the wall-climbing robot in the prior map.

[0039] Step S24: When the onboard camera detects the features of the QR code preset on the body of the wall-climbing robot, it immediately calculates its three-dimensional spatial position relative to the drone through the pose estimation algorithm. At the same time, combined with the drone's own positioning data, the coordinates of the wall-climbing robot are converted into the global map, and the positioning results are sent to the ground control unit through the communication link for cross-verification. If there is a deviation between the lidar point cloud and the visual positioning, the ground control unit adopts a fusion strategy to correct the final initial coordinates of the wall-climbing robot to ensure that the position error of the two in the unified coordinate system is less than the threshold.

[0040] As a further solution of the present invention: the specific steps in step S3 include:

[0041] Step S31: Fusing and registering the laser point cloud and image data of the UAV and the wall-climbing robot, and updating the global map in real time;

[0042] Step S32: Using the fused map, the wall-climbing robot optimizes its climbing trajectory based on the global path suggestion provided by the ground control unit and using a dynamic path planning algorithm.

[0043] As a further solution of the present invention: the specific steps in step S4 include:

[0044] The drone uses a trajectory prediction control algorithm to adjust the flight altitude and angle, maintain pitch angle tracking within a preset distance, and combines a path planning algorithm to bypass sudden obstacles.

[0045] Compared with the prior art, the present invention has the following technical effects:

[0046] By employing the aforementioned technical solution, the present invention utilizes a drone-robot collaborative perception architecture, innovatively integrating high-altitude bird's-eye view with near-field wall-based perception data to construct a multi-resolution 3D semantic map. The drone's onboard LiDAR and camera utilize a geometry-texture coupled feature extraction algorithm, combined with a dynamic block segmentation strategy and bag-of-visual-words closed-loop detection. This solves the problem of feature degradation caused by the limited field of view of a single sensor in complex wall environments, thereby improving the accuracy of environmental modeling.

[0047] This invention utilizes a path planning system based on a hierarchical decision-making architecture, combining global path planning for the drone with local obstacle avoidance for the robot. The drone uses a trajectory prediction control algorithm to dynamically maintain its optimal observation position. The dynamic path planning algorithm deployed on the wall-climbing robot autonomously selects the optimal navigation target point and plans the navigation path, improving the system's applicability and robustness for navigation tasks in unknown environments.

[0048] This invention builds a complete intelligent operation system for wall-climbing robots through innovative designs such as multi-agent collaborative perception, hierarchical decision optimization, and multimodal data fusion. It uses the ground main control unit as the core hub to achieve multi-source data fusion, global path planning, and real-time collaborative scheduling, significantly improving the system response speed and operation reliability. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings:

[0050] Figure 1 A schematic diagram of the steps of the positioning and path planning method for a wall-climbing robot according to an embodiment disclosed in this application;

[0051] Figure 2 A schematic diagram of a system for positioning and path planning of a wall-climbing robot according to an embodiment disclosed in this application;

[0052] Figure 3 A flowchart for constructing a three-dimensional map of the working environment according to the embodiment disclosed in this application;

[0053] Figure 4 This is a flow chart of the visual word bag loop closure detection model of the embodiment disclosed in this application;

[0054] Figure 5 This is a flow chart of the UAV tracking and positioning wall-climbing robot according to the disclosed embodiment of the present application;

[0055] Figure 6 This is a flowchart of the positioning and path planning of the wall-climbing robot according to the embodiment disclosed in this application. DETAILED DESCRIPTION

[0056] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0057] Please refer to Figure 1 In an embodiment of the present invention, a positioning and path planning method for a wall-climbing robot based on UAV collaboration is provided.

[0058] In this embodiment, Figure 2 As shown, the system equipment includes a UAV, a wall-climbing robot, and a ground control unit. The UAV is equipped with an embedded PC, a lidar module, and a camera module for positioning and environmental perception of the wall-climbing robot.

[0059] The wall-climbing robot is equipped with an embedded PC, an inertial measurement unit, a lidar module, and a camera module, and the ground control unit is equipped with a main control module and a signal relay module;

[0060] The process of building a 3D map uses methods such as laser vision fusion SLAM, dynamic surface segmentation, and visual word bag closed loop detection;

[0061] The specific steps include:

[0062] Step S1: Based on the point cloud and image data collected by the lidar and camera, a 3D map in the global coordinate system is constructed and updated through segmentation, registration, pose graph optimization, and visual loop closure detection. The specific steps include:

[0063] like Figure 3 As shown, the figure is a flowchart for constructing a three-dimensional map of the working environment;

[0064] Step S11: Collecting lidar point cloud and image data respectively through the lidar and camera carried by the drone;

[0065] Step S12: Perform feature extraction, matching, and local motion estimation on the pre-processed data, segment the point cloud into local sub-blocks, and align adjacent sub-blocks to the local coordinate system using a point cloud registration algorithm and a feature matching algorithm to form a preliminary coherent local map. This specifically includes:

[0066] In the first step, the geometric characteristics of the lidar point cloud are obtained through curvature analysis, normal vector clustering or RANSAC plane fitting methods, and the texture information in the image is obtained through weld recognition, edge detection and semantic segmentation network enhanced feature extraction;

[0067] In the second step, the ICP algorithm and Kd tree model are used to complete the feature matching of the lidar scanning data, extract the image ORB features, use the FLANN method to achieve feature matching, and use the RANSAC algorithm to filter out false matches;

[0068] The third step is to obtain the depth information of the pixels in the image and complete the local motion estimation through the ICP method;

[0069] The fourth step combines the geometric characteristics of the point cloud with the texture information of the image to achieve refined segmentation that coordinates geometric structure and visual features. At the same time, a dynamic segmentation strategy is introduced to adjust the local segmentation based on the real-time motion trajectory of the drone to ensure map continuity during the scanning process.

[0070] In the fifth step, the multimodal features extracted from the current frame are jointly optimized with the historical sub-blocks through factor graphs. By minimizing the reprojection error of the edge / plane features of the laser point cloud and the geometric consistency error of the visual features, a local sub-map containing geometric structure, texture information and semantic labels is generated.

[0071] In the sixth step, after each sub-map passes the spatiotemporal consistency test, its central pose, feature distribution, and boundary range are recorded to provide input for subsequent pose graph construction and global optimization.

[0072] Step S13: Use the sub-blocks as nodes and the transformation relationships between adjacent sub-blocks as edges to construct a pose graph, identify overlapping areas through visual word bags, and add closed-loop constraints, specifically including:

[0073] In the first step, the center position and pose of each submap are initialized as pose graph nodes, and the relative pose transformation of adjacent submaps obtained by ICP registration and visual feature matching is added to the graph as an edge constraint;

[0074] The second step is Figure 4 As shown in the figure, it is a flow chart of the visual word bag loop detection model. The visual word bag model is used to perform similarity retrieval on the ORB feature descriptors extracted from the submap, and historical submaps with overlapping areas are screened out as closed loop candidates. When a closed loop candidate is detected, a geometric consistency verification framework based on the PnP algorithm is adopted, and RANSAC is used to eliminate mismatched feature points and calculate the relative pose transformation of the candidate closed loop. If its reprojection error is lower than the preset threshold and the spatial translation is within the IMU error range, the closed loop constraint is added as a new edge to the pose graph.

[0075] In the third step, the global pose graph is iteratively optimized using the g2o optimization framework for global consistency optimization.

[0076] Step S14: Using the optimized pose graph, coordinate system alignment is performed to transform all sub-blocks into a unified global coordinate system;

[0077] Step S15: Merge all sub-block point clouds in the global coordinate system into a complete scene point cloud, convert the point cloud into a spatial index structure according to the resolution, and obtain and update the global map.

[0078] In this embodiment, in step S1, the visual word bag method is used to perform visual closed-loop detection, and the K-means clustering method is used to train a binary offline word bag containing various image features. The process of training the visual word bag is: clustering each image frame according to its features and descriptors, and after clustering, numbering each image frame to generate a visual word bag; in the closed-loop or key frame detection process, the current frame image information is converted into a word bag vector that can be recognized by the word bag, and similar image frames are searched in the word bag through similarity measurement. If they exist, it is determined that a closed-loop or key frame match is detected, and then the relative motion between the two frames is calculated to optimize the current posture.

[0079] In this embodiment, the process of transforming all sub-blocks into a unified global coordinate system in step S1 is as follows:

[0080] The first step is to traverse all submaps and extract their central pose parameters based on the optimized submap node pose. The rigid transformation matrix of each submap from the local coordinate system to the global coordinate system is calculated through coordinate transformation matrix operations.

[0081] In the second step, the corresponding transformation matrix is ​​applied to the point cloud data of each submap to uniformly convert its coordinates, normal vectors, and semantic labels to the global coordinate system. At the same time, the point cloud registration residuals of adjacent submaps in the overlapping area are detected. If the mean residual exceeds the preset threshold, the global registration fine-tuning is triggered.

[0082] In the third step, multi-resolution point cloud registration is used to perform non-rigid transformation optimization on the sub-map point cloud in the conflict area, and its transformation matrix is ​​iteratively updated until the residual converges.

[0083] Step S2: Based on the constructed three-dimensional map in the global coordinate system, the UAV is used to identify the QR code of the wall-climbing robot and combine its own positioning with the cross-verification and fusion correction of the ground control unit to obtain the precise positioning information of the wall-climbing robot in the global three-dimensional map;

[0084] like Figure 5 As shown, the figure shows the flow chart of UAV tracking and positioning of wall-climbing robot;

[0085] In this embodiment, the specific steps in step S2 include:

[0086] Step S21: When starting up, the ground control unit first loads a pre-stored 3D map, unifies the sensor data of the UAV and the wall-climbing robot into the global coordinate system of the map through a coordinate conversion protocol, and uses a sensor calibration tool to ensure that the external parameters of each sensor are aligned;

[0087] Step S22: After the wall-climbing robot is started, its onboard laser radar and camera begin to scan the surrounding environment, extract wall feature points, match the real-time features with the key frames in the 3D map through the multi-sensor fusion SLAM algorithm, calculate the initial pose in combination with graph optimization technology, and use IMU data to compensate for motion distortion to complete the initial positioning in the map;

[0088] Step S23: The UAV is lifted off from the take-off point to a predetermined height. The onboard camera scans the operating area using a preset visual recognition module and selects targets within the visual range based on the expected position range of the wall-climbing robot in the prior map.

[0089] Step S24: When the onboard camera detects the features of the QR code preset on the body of the wall-climbing robot, it immediately calculates its three-dimensional spatial position relative to the drone through the pose estimation algorithm. At the same time, combined with the drone's own positioning data, the coordinates of the wall-climbing robot are converted into the global map, and the positioning results are sent to the ground control unit through the communication link for cross-verification. If there is a deviation between the lidar point cloud and the visual positioning, the ground control unit adopts a fusion strategy to correct the final initial coordinates of the wall-climbing robot to ensure that the position error of the two in the unified coordinate system is less than the threshold.

[0090] Step S3: Synchronously integrating the perception data of the UAV and the wall-climbing robot to update the map; the wall-climbing robot optimizes the climbing trajectory using a dynamic path planning algorithm based on the obtained positioning information and the global planning path of the ground control unit;

[0091] In this embodiment, Figure 6 As shown in the figure, it is a flowchart of positioning and path planning of the wall-climbing robot. The specific steps in step S3 include:

[0092] Step S31: Fusing and registering the laser point cloud and image data of the UAV and the wall-climbing robot, and updating the global map in real time, specifically, updating the octree map in real time;

[0093] Step S32: Based on the fused map, the wall-climbing robot optimizes its climbing trajectory using a dynamic path planning algorithm based on the global path suggestion provided by the ground control unit, specifically using the dynamic window algorithm (DWA) to optimize its climbing trajectory.

[0094] Step S4: Based on the positioning information and environmental information of the wall-climbing robot, the UAV uses a trajectory prediction control algorithm to adjust its posture to maintain a preset distance tracking, and at the same time avoids sudden obstacles in real time through path planning.

[0095] In this embodiment, the specific steps in step S4 include:

[0096] The drone uses a trajectory prediction control algorithm to adjust the flight altitude and angle, maintaining pitch angle tracking within a preset distance, such as maintaining a 30° pitch angle tracking within a distance of 5-10 meters, while combining the path planning RRT* algorithm to bypass sudden obstacles.

[0097] In a specific embodiment, the predictive control algorithm may adopt a model predictive control (MPC) algorithm, when the UAV maintains pitch angle tracking within a preset distance, for example, it may be set to maintain a 30° pitch angle tracking within a distance of 5-10 meters.

[0098] In this embodiment, software and hardware collaboration is achieved through the ROS framework. An adaptive self-organizing network is used at the communication level. Signal relay units are deployed in visual blind spots, giving priority to transmitting positioning data and compressing point cloud images to ensure that key instructions can be delivered with low latency.

[0099] When the wall-climbing robot encounters signal interruption or positioning loss, the drone switches to pure visual tracking mode, providing temporary coordinate references through the target detection algorithm, while the wall-climbing robot enables the local cache path to continue operating until communication is restored.

[0100] Specifically, the entire system achieves software and hardware collaboration through the ROS framework, and an adaptive Mesh network is used at the communication level. Signal relay units are deployed in visual blind spots, prioritizing the transmission of positioning data and compressing point cloud images to ensure that key instructions can be delivered with low latency. When the wall-climbing robot encounters signal interruption or positioning loss, the drone switches to pure visual tracking mode, providing temporary coordinate references through YOLO target detection, while the wall-climbing robot enables the local cache path to continue operating until communication is restored.

[0101] Although the embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations may be made to these embodiments without departing from the principles and spirit of the present invention. The scope of the present invention is defined by the appended claims and their equivalents, and all should be included within the scope of protection of the present invention.

Claims

1. A positioning and path planning method for a wall-climbing robot based on UAV collaboration, characterized in that: The following steps are involved: Step S1: Based on the point cloud and image data collected by the lidar and camera, a three-dimensional map in the global coordinate system is constructed and updated through segmentation, registration, pose graph optimization, and visual loop closure detection; Step S2: Based on the constructed three-dimensional map in the global coordinate system, the UAV is used to identify the QR code of the wall-climbing robot and combine its own positioning with the cross-verification and fusion correction of the ground control unit to obtain the precise positioning information of the wall-climbing robot in the global three-dimensional map; Step S3: Synchronously integrating the perception data of the UAV and the wall-climbing robot to update the map; the wall-climbing robot optimizes the climbing trajectory using a dynamic path planning algorithm based on the obtained positioning information and the global planning path of the ground control unit; Step S4: Based on the positioning information and environmental information of the wall-climbing robot, the UAV uses a trajectory prediction control algorithm to adjust its posture to maintain a preset distance tracking, and at the same time avoids sudden obstacles in real time through path planning.

2. A method for positioning and path planning of a wall-climbing robot based on cooperation with drones according to claim 1, characterized in that: The specific steps in step S1 include: Step S11: Collecting lidar point cloud and image data respectively through the lidar and camera carried by the drone; Step S12: perform feature extraction, matching, and local motion estimation on the preprocessed data, segment the point cloud into local sub-blocks, and align adjacent sub-blocks to the local coordinate system using a point cloud registration algorithm and a feature matching algorithm to form a preliminary coherent local map; Step S13: Use the sub-blocks as nodes and the transformation relationships between adjacent sub-blocks as edges to construct a pose graph, identify overlapping areas through the visual word bag, and add closed-loop constraints; Step S14: Using the optimized pose graph, coordinate system alignment is performed to transform all sub-blocks into a unified global coordinate system; Step S15: Merge all sub-block point clouds in the global coordinate system into a complete scene point cloud, convert the point cloud into a spatial index structure according to the resolution, and obtain and update the global map.

3. The method for positioning and path planning of a wall-climbing robot based on cooperation with unmanned aerial vehicles according to claim 2, characterized in that: The process of dividing the sub-blocks and forming the local map in step S1 is as follows: The first step is to enhance data feature extraction capabilities through geometric feature analysis methods, plane fitting algorithms, or semantic segmentation networks to obtain the geometric characteristics of the lidar point cloud and texture information in the image. The second step is to use the point cloud registration algorithm and spatial index model to complete the feature matching of the lidar scanning data, extract the image feature descriptor, use the fast nearest neighbor approximation search method to achieve feature matching, and use the false match filtering algorithm to filter out false matches; The third step is to obtain the depth information of the pixels in the image and complete the pose estimation through the local motion estimation algorithm; Step 4: Combining the geometric characteristics of the point cloud with the texture information of the image, we achieve refined segmentation that coordinates geometric structure and visual features. At the same time, we introduce a dynamic segmentation strategy to adjust the local segmentation based on the real-time motion trajectory of the drone to ensure map continuity. Step 5: Jointly optimize the multimodal features extracted from the current frame with the historical sub-blocks. By minimizing the reprojection error of the laser point cloud features and the geometric consistency error of the visual features, a fusion is generated to generate a local sub-map containing geometric structure, texture information, and semantic labels. Step 6: After each sub-map passes the spatiotemporal consistency test, its central position, feature distribution, and boundary range are recorded to provide input for subsequent pose graph construction and global optimization.

4. The method for positioning and path planning of a wall-climbing robot based on cooperation with unmanned aerial vehicles according to claim 2, characterized in that: The process of constructing the pose graph and the visual word bag closed loop detection in step S1 is as follows: The first step is to initialize the center position and posture of each submap as a pose graph node, and add the relative pose transformation of adjacent submaps obtained by registration algorithm and visual feature matching as edge constraints into the graph; In the second step, the feature descriptors extracted from the submaps are similarly searched using the bag-of-visual-words model, and historical submaps with overlapping areas are selected as closed-loop candidates. When a closed-loop candidate is detected, a geometric consistency verification framework is used to remove mismatched feature points using RANSAC and calculate the relative pose transformation of the candidate closed loop. If the error is lower than a preset threshold and the spatial translation is within the inertial measurement error range, the closed-loop constraint is added as a new edge to the pose graph. The third step is to use the graph optimization algorithm to iteratively optimize the global pose graph.

5. The method for positioning and path planning of a wall-climbing robot based on cooperation with unmanned aerial vehicles according to claim 2, characterized in that: In step S1, a visual word bag method is used to perform visual closed-loop detection, and a binary offline word bag containing various image features is trained using a K-means clustering method. The process of training the visual word bag is as follows: each image frame is clustered according to its features and descriptors, and after clustering, each image frame is numbered to generate a visual word bag; during the closed-loop or keyframe detection process, the current frame image information is converted into a word bag vector that can be recognized by the word bag, and similar image frames are searched in the word bag through a similarity metric. If they exist, it is determined that a closed-loop or keyframe match is detected, and then the relative motion between the two frames is calculated to optimize the current posture.

6. The method for positioning and path planning of a wall-climbing robot based on cooperation with unmanned aerial vehicles according to claim 2, characterized in that: The process of transforming all sub-blocks into a unified global coordinate system in step S1 is as follows: The first step is to traverse all submaps and extract their central pose parameters based on the optimized submap node pose. The rigid transformation matrix of each submap from the local coordinate system to the global coordinate system is calculated through coordinate transformation matrix operations. In the second step, the corresponding transformation matrix is ​​applied to the point cloud data of each submap to uniformly convert its coordinates, normal vectors, and semantic labels to the global coordinate system. At the same time, the point cloud registration residuals of adjacent submaps in the overlapping area are detected. If the mean residual exceeds the preset threshold, the global registration fine-tuning is triggered. In the third step, multi-resolution point cloud registration is used to perform non-rigid transformation optimization on the sub-map point cloud in the conflict area, and its transformation matrix is ​​iteratively updated until the residual converges.

7. The method for positioning and path planning of a wall-climbing robot based on cooperation with unmanned aerial vehicles according to claim 2, characterized in that: The ROS framework enables software and hardware collaboration. An adaptive self-organizing network is used for communication. Signal relay units are deployed in blind spots, prioritizing the transmission of positioning data and compressing point cloud images to ensure that key commands can be delivered with low latency. When the wall-climbing robot encounters signal interruption or positioning loss, the drone switches to pure visual tracking mode, providing temporary coordinate references through the target detection algorithm, while the wall-climbing robot enables the local cache path to continue operating until communication is restored.

8. The method for positioning and path planning of a wall-climbing robot based on cooperation with unmanned aerial vehicles according to claim 1, characterized in that: The specific steps in step S2 include: Step S21: When starting up, the ground control unit first loads a pre-stored 3D map, unifies the sensor data of the UAV and the wall-climbing robot into the global coordinate system of the map through a coordinate conversion protocol, and uses a sensor calibration tool to ensure that the external parameters of each sensor are aligned; Step S22: After the wall-climbing robot is started, its onboard laser radar and camera begin to scan the surrounding environment, extract wall feature points, match the real-time features with the key frames in the 3D map through the multi-sensor fusion SLAM algorithm, calculate the initial pose in combination with graph optimization technology, and use IMU data to compensate for motion distortion to complete the initial positioning in the map; Step S23: The UAV is lifted off from the take-off point to a predetermined height. The onboard camera scans the operating area using a preset visual recognition module and selects targets within the visual range based on the expected position range of the wall-climbing robot in the prior map. Step S24: When the onboard camera detects the features of the QR code preset on the body of the wall-climbing robot, it immediately calculates its three-dimensional spatial position relative to the drone through the pose estimation algorithm. At the same time, combined with the drone's own positioning data, the coordinates of the wall-climbing robot are converted into the global map, and the positioning results are sent to the ground control unit through the communication link for cross-verification. If there is a deviation between the lidar point cloud and the visual positioning, the ground control unit adopts a fusion strategy to correct the final initial coordinates of the wall-climbing robot to ensure that the position error of the two in the unified coordinate system is less than the threshold.

9. The method for positioning and path planning of a wall-climbing robot based on cooperation with unmanned aerial vehicles according to claim 1, characterized in that: The specific steps in step S3 include: Step S31: Fusing and registering the laser point cloud and image data of the UAV and the wall-climbing robot, and updating the global map in real time; Step S32: Using the fused map, the wall-climbing robot optimizes its climbing trajectory based on the global path suggestion provided by the ground control unit and using a dynamic path planning algorithm.

10. The method for positioning and path planning of a wall-climbing robot based on cooperation with unmanned aerial vehicles according to claim 1, characterized in that: The specific steps in step S4 include: The drone uses a trajectory prediction control algorithm to adjust the flight altitude and angle, maintain pitch angle tracking within a preset distance, and combines a path planning algorithm to bypass sudden obstacles.

Images