Wall-climbing robot space positioning method and device based on visual recognition

Abstract

The application provides a wall-climbing robot space positioning method and device based on visual recognition, and belongs to the technical field of robot positioning. The method comprises the following steps: setting an Aruco marker with a preset size and number on a wall-climbing robot body, taking the wall-climbing robot body as a reference, constructing a body coordinate system, determining a marker coordinate of the Aruco marker in the body coordinate system, and gradually performing coordinate transformation based on the marker coordinate to obtain a three-dimensional space coordinate of the wall-climbing robot in a ground coordinate system. The wall-climbing robot is path planned and motion controlled according to the three-dimensional space coordinate. The method solves the problems of the traditional positioning mode of the existing wall-climbing robot, especially the difficulty in playing a role in indoor, steel structure or closed environments, and the problems of large cumulative error and insufficient positioning accuracy.

Description

Technical Field

[0001] This invention belongs to the field of robot positioning technology, and relates to a spatial positioning method and device for a wall-climbing robot based on visual recognition. Background Technology

[0002] Wall-climbing robots are widely used in wind turbine tower inspection, ship hull maintenance, tank wall maintenance, and steel structure inspection. Traditional positioning methods such as GPS, inertial navigation, and odometers are severely limited by the environment, especially in indoor, steel structure, or enclosed environments, and suffer from large cumulative errors and insufficient positioning accuracy. Therefore, there is an urgent need for a positioning technology that is suitable for complex environments, has high precision, and good real-time performance. Summary of the Invention

[0003] The purpose of this invention is to address the aforementioned problems in existing technologies by proposing a spatial positioning method for wall-climbing robots based on visual recognition.

[0004] The objective of this invention can be achieved through the following technical solutions:

[0005] A spatial localization method for a wall-climbing robot based on vision recognition, comprising:

[0006] Set Aruco markers of preset size and number on the body of the wall-climbing robot, and construct a body coordinate system based on the body of the wall-climbing robot to determine the marker coordinates of the Aruco markers in the body coordinate system;

[0007] The Aruco marker was captured by a visual camera mounted on the gimbal, and the real-time pitch and yaw angles of the gimbal were read during the capture process.

[0008] Using the visual camera as a reference, a camera coordinate system is constructed. The real-time pitch angle and real-time yaw angle of the gimbal are analyzed according to the PnP algorithm to determine the coordinates of the Aruco marker in the camera coordinate system.

[0009] By measuring the straight-line distance between the gimbal and the wall-climbing robot using a laser rangefinder, and analyzing the coordinates of the Aruco marker in the camera coordinate system based on the straight-line distance, the real-time pitch angle, and the real-time yaw angle, the three-dimensional spatial coordinates of the wall-climbing robot in the ground coordinate system with the ground as the reference are calculated.

[0010] As an optional embodiment of the present invention, a camera coordinate system is constructed based on the visual camera, and the real-time pitch angle and real-time yaw angle of the gimbal are analyzed according to the PnP algorithm to determine the coordinates of the Aruco marker in the camera coordinate system, including:

[0011] Obtain the initial pitch angle and initial yaw angle of the gimbal in the camera coordinate system;

[0012] The initial pitch angle, initial yaw angle, real-time pitch angle, and real-time yaw angle are analyzed using PnP to obtain the rotation matrix and displacement vector;

[0013] The coordinates of the Aruco marker in the camera coordinate system are determined by performing coordinate transformation calculations based on the rotation matrix and the displacement vector.

[0014] As an optional embodiment of the present invention, by measuring the straight-line distance between the gimbal and the wall-climbing robot using a laser rangefinder mounted on the gimbal, and analyzing the coordinates of the Aruco marker in the camera coordinate system based on the straight-line distance, the real-time pitch angle, and the real-time yaw angle, the three-dimensional spatial coordinates of the wall-climbing robot in a ground coordinate system with the ground as the reference are calculated, including:

[0015] The spatial coordinates of the visual camera in the ground coordinate system are calculated based on the straight-line distance, the real-time pitch angle, and the real-time yaw angle.

[0016] The spatial coordinates of the vision camera and the coordinates of the Aruco marker in the camera coordinate system are transformed to obtain the three-dimensional spatial coordinates of the wall-climbing robot in the ground coordinate system with the ground as the reference.

[0017] As an optional embodiment of the present invention, by capturing the Aruco marker using a visual camera mounted on a gimbal, the invention further includes:

[0018] The position of the wall-climbing robot is initially detected using the YOLO algorithm, and a captured image is generated.

[0019] Using the coordinates of the center pixel of the captured image as a reference, the degree of deviation is calculated in the captured image based on the coordinates of the Aruco marker in the coordinate system where the center pixel is located, and the gimbal movement is controlled according to the degree of deviation.

[0020] As an optional embodiment of the present invention, the Aruco marker includes the coordinates of the center pixel in the coordinate system, comprising:

[0021] Extract the pose information of the Aruco marker from the captured image;

[0022] Obtain the IMU information of the wall-climbing robot;

[0023] The pose of the wall-climbing robot is estimated based on the pose information and the IMU information, and the coordinates of the Aruco marker in the coordinate system of the center pixel are determined.

[0024] As an optional embodiment of the present invention, the pose information includes the center position, rotation angle, and identification ID of the Aruco marker;

[0025] The IMU information includes the three-dimensional acceleration and three-dimensional angular velocity of the wall-climbing robot.

[0026] As an optional embodiment of the present invention, it further includes: during the process of controlling the movement of the gimbal according to the degree of deviation, using a PID feedback control algorithm to make the Aruco marker located at the center position of the visual camera's field of view.

[0027] As an optional embodiment of the present invention, the coordinates of the Aruco marker in the body coordinate system do not change with the movement of the wall-climbing robot.

[0028] The present invention also provides a spatial positioning device for a wall-climbing robot based on visual recognition, comprising:

[0029] The Aruco marking module is set up to set Aruco markings of preset size and number on the body of the wall-climbing robot, and to construct a body coordinate system based on the body of the wall-climbing robot to determine the marking coordinates of the Aruco markings in the body coordinate system.

[0030] The capture module is used to capture the Aruco marker by setting a visual camera on the gimbal, and to read the real-time pitch angle and real-time yaw angle of the gimbal during the capture process;

[0031] The marker coordinate transformation module is used to construct a camera coordinate system based on the visual camera, analyze the real-time pitch angle and real-time yaw angle of the gimbal according to the PnP algorithm, and determine the coordinates of the Aruco marker in the camera coordinate system.

[0032] The module for calculating three-dimensional spatial coordinates is used to measure the straight-line distance between the gimbal and the wall-climbing robot by setting a laser rangefinder on the gimbal, and to analyze the coordinates of the Aruco marker in the camera coordinate system based on the straight-line distance, the real-time pitch angle and the real-time yaw angle, and to calculate the three-dimensional spatial coordinates of the wall-climbing robot in the ground coordinate system with the ground as the reference.

[0033] The present invention also provides an electronic device, comprising:

[0034] processor;

[0035] Memory used to store processor-executable instructions;

[0036] The processor is configured to implement the aforementioned vision-based spatial positioning method for wall-climbing robots when executing the executable instructions.

[0037] Compared with existing technologies, this invention combines the advantages of visual recognition technology and laser ranging technology. It is not limited by GPS, inertial navigation or odometer, and is particularly suitable for robot positioning in enclosed environments, steel structure environments and other areas where GPS signals cannot cover. It has the characteristics of strong real-time performance, high accuracy and strong anti-interference ability, which greatly improves the efficiency and safety of wall-climbing robots in actual operation and is applicable to a variety of application scenarios in industrial sites. Attached Figure Description

[0038] Figure 1 This is a flowchart of a visual recognition-based spatial positioning method for a wall-climbing robot according to an embodiment of this application;

[0039] Figure 2 This is a schematic diagram of an embodiment of this application;

[0040] Figure 3 This is a diagram of a vision-based wall-climbing robot spatial positioning device according to an embodiment of this application. Detailed Implementation

[0041] The following are specific embodiments of the present invention, which are described in conjunction with the accompanying drawings. However, the present invention is not limited to these embodiments.

[0042] It should be noted that all directional indications (such as up, down, left, right, front, back, etc.) in the embodiments of the present invention are only used to explain the relative positional relationship and movement of each component in a certain specific posture (as shown in the figure). If the specific posture changes, the directional indication will also change accordingly.

[0043] Example 1

[0044] To address the positioning difficulties of wall-climbing robots indoors, in steel structures, or in enclosed spaces, such as... Figure 1 This embodiment proposes a spatial localization method for a wall-climbing robot based on visual recognition, including:

[0045] S1, set Aruco marks of preset size and number on the body of the wall-climbing robot, and construct a body coordinate system based on the body of the wall-climbing robot to determine the mark coordinates of the Aruco marks in the body coordinate system;

[0046] S2, The Aruco marker is captured by a visual camera set on the gimbal, and the pitch and yaw angles of the gimbal are read in real time during the capture process;

[0047] S3. Using the visual camera as a reference, construct a camera coordinate system, analyze the real-time pitch angle and real-time yaw angle of the gimbal according to the PnP algorithm, and determine the coordinates of the Aruco marker in the camera coordinate system.

[0048] S4. By setting a laser rangefinder on the gimbal to measure the straight-line distance between the gimbal and the wall-climbing robot, the coordinates of the Aruco marker in the camera coordinate system are analyzed based on the straight-line distance, the real-time pitch angle and the real-time yaw angle, and the three-dimensional spatial coordinates of the wall-climbing robot in the ground coordinate system with the ground as the reference are calculated.

[0049] In this embodiment, an Aruco marker of a preset size and number is affixed to the body of the wall-climbing robot, ensuring the marker is clearly identifiable. An Aruco marker is a square encoding a binary matrix, providing stable feature points in an image to reveal the projection relationship between the wall-climbing robot in the two-dimensional and three-dimensional worlds, thus determining the robot's position and orientation in three-dimensional space. This embodiment uses coordinates to represent this projection relationship. Therefore, after affixing the Aruco marker, a body coordinate system is constructed based on the robot's body, and the coordinates of the Aruco marker within this system are determined and denoted as marker coordinates for subsequent projection relationship analysis.

[0050] like Figure 2 As shown, a dual-axis rotatable gimbal, including a pitch axis and a yaw axis, is fixedly installed on the ground. A high-precision motor drive system drives the two degrees of freedom motion mechanisms, ensuring the gimbal's movement in all directions. An angle feedback sensor is also equipped to ensure angle measurement accuracy of 0.01°. During gimbal movement, the pitch and yaw angles are transmitted to a ground computer for analysis, revealing the wall-climbing robot's motion.

[0051] The gimbal is also equipped with a vision camera and a laser rangefinder. The vision camera uses a high-resolution industrial-grade camera to ensure accurate identification of Aruco markers at long distances. The laser rangefinder uses an industrial-grade high-precision ranging sensor with a ranging error within ±0.1mm. After mounting the gimbal on a stable, fixed support on the ground, initial position calibration is performed to ensure the stability of the positioning reference. Since the gimbal motion controls the vision camera to capture Aruco markers, the gimbal and vision camera can be considered as a single unit. The vision camera can be used as a reference to construct a camera coordinate system. When the gimbal moves, the camera coordinate system rotates to a certain extent, causing changes in the orientation of the coordinate axes. Based on the real-time pitch and yaw angles of the gimbal obtained from the angle feedback sensor, the fixed marker coordinates in the fuselage coordinate system are calculated and converted into their corresponding coordinates in the camera coordinate system, thus determining the coordinates of the Aruco marker in the camera coordinate system.

[0052] In this embodiment, the ground computer indirectly monitors the wall-climbing robot's movement via a gimbal, constructs a fusion vision-laser ranging multi-point positioning model, and accurately calculates the robot's real-time 3D spatial coordinates (X, Y, Z), thereby controlling the robot's path planning and movement. In this process, a ground coordinate system is established based on the ground. The transformation relationship between the ground coordinate system and the camera coordinate system is calculated based on the straight-line distance between the gimbal and the wall-climbing robot measured by the laser rangefinder, the gimbal's real-time pitch angle, and real-time yaw angle. This determines the robot's coordinates in the ground coordinate system, allowing the ground computer to control the robot's motion.

[0053] Preferably, a camera coordinate system is constructed based on the visual camera. The real-time pitch angle and real-time yaw angle of the gimbal are analyzed according to the PnP algorithm to determine the coordinates of the Aruco marker in the camera coordinate system. This includes: obtaining the initial pitch angle and initial yaw angle of the gimbal in the camera coordinate system; performing PnP analysis on the initial pitch angle, initial yaw angle, real-time pitch angle, and real-time yaw angle to obtain a rotation matrix and a displacement vector; and performing coordinate transformation calculation on the marker coordinates according to the rotation matrix and the displacement vector to determine the coordinates of the Aruco marker in the camera coordinate system.

[0054] When the gimbal moves the vision camera, the camera coordinate system also changes. Before determining the coordinates of the Aruco marker in the camera coordinate system, the transformation relationship before and after the camera coordinate system change is calculated. Using the real-time pitch angle, real-time yaw angle, initial pitch angle, and initial yaw angle of the gimbal, the rotation matrix R and displacement vector t are calculated using the PnP algorithm. The transformation relationship between the initial camera coordinate system and the real-time camera coordinate system can be determined using the rotation matrix R and displacement vector t. Let the real-time coordinates of the Aruco marker in the real-time camera coordinate system be Pc=[Xc, Yc, Zc]^T, where T is the transpose. Let the marker coordinates be Pw. Pw does not change in the body coordinate system as the climbing robot moves, but it changes in the camera coordinate system with the movement of the climbing robot and multiple changes in the camera coordinate system. Then, Pc=R*Pw+t is calculated using the rotation matrix R and displacement vector t to obtain the coordinates of the Aruco marker in the camera coordinate system.

[0055] Preferably, the straight-line distance between the gimbal and the wall-climbing robot is measured by setting a laser rangefinder on the gimbal. The coordinates of the Aruco marker in the camera coordinate system are analyzed based on the straight-line distance, the real-time pitch angle, and the real-time yaw angle. The three-dimensional spatial coordinates of the wall-climbing robot in the ground coordinate system with the ground as the reference are calculated. This includes: calculating the spatial coordinates of the vision camera in the ground coordinate system based on the straight-line distance, the real-time pitch angle, and the real-time yaw angle; performing coordinate transformation calculations on the spatial coordinates of the vision camera and the coordinates of the Aruco marker in the camera coordinate system; and obtaining the three-dimensional spatial coordinates of the wall-climbing robot in the ground coordinate system with the ground as the reference.

[0056] Similarly, when the gimbal moves the vision camera, the camera coordinate system will also change, and consequently, the camera coordinate system will also change relative to the ground coordinate system. Let the straight-line distance measured by the rangefinder be d, and the spatial coordinates Pg of the vision camera in the ground coordinate system can be calculated using the following formula:

[0057]

[0058] Given the coordinates of the Aruco marker in the camera coordinate system and the spatial coordinates of the vision camera in the ground coordinate system, the three-dimensional position P_robot of the wall-climbing robot body in the ground coordinate system is calculated as: Probot = Pg + R*Pc, where R*Pc represents the coordinates of the Aruco marker in the camera coordinate system after the camera coordinate system changes.

[0059] Preferably, capturing the Aruco marker by setting a vision camera on the gimbal further includes: initially detecting the position of the wall-climbing robot using the YOLO algorithm, generating a captured image, using the coordinates of the center pixel of the captured image as a reference, calculating the degree of deviation in the captured image based on the coordinates of the Aruco marker in the coordinate system where the center pixel is located, and controlling the movement of the gimbal based on the degree of deviation.

[0060] The YOLO algorithm is used to detect and view the position information of the wall-climbing robot in real time. A vision camera captures images of the Aruco marker on the robot in real time. To keep the robot centered in the captured image, the vision camera needs to continuously track the robot and keep it at the center of the viewpoint. In this embodiment, a coordinate system is established at the center pixel of the captured image, the coordinates of the Aruco marker in this coordinate system are determined, and the distance between these coordinates and the center pixel is calculated to determine the degree of deviation. Based on the degree of deviation, the gimbal moves the vision camera up, down, left, and right to keep the Aruco marker at the center of the vision camera's viewpoint.

[0061] Preferably, the coordinates of the Aruco marker in the coordinate system where the center pixel is located include: extracting the pose information of the Aruco marker from the captured image, obtaining the IMU information of the wall-climbing robot, estimating the pose of the wall-climbing robot based on the pose information and the IMU information, and determining the coordinates of the Aruco marker in the coordinate system where the center pixel is located.

[0062] Since different poses of the wall-climbing robot in the same coordinate system will result in different coordinates of the Aruco marker, after the captured image is transmitted to the ground computer, the pose information of the Aruco marker is extracted using image processing algorithms. Combined with the acquired IMU information of the wall-climbing robot, the pose of the wall-climbing robot is estimated. After determining the pose of the wall-climbing robot, the position of pasting the Aruco marker is determined, and the coordinates of the Aruco marker in the coordinate system of the center pixel can be obtained.

[0063] Preferably, the pose information includes the center position, rotation angle, and identification ID of the Aruco marker, and the IMU information includes the three-dimensional acceleration and three-dimensional angular velocity of the wall-climbing robot.

[0064] When affixing Aruco tags, they should be placed in a location that is clearly identifiable and easily reflects the state of the wall-climbing robot. In practice, there are multiple wall-climbing robots used for detection, and each wall-climbing robot has a different Aruco tag. Therefore, the posture of the corresponding wall-climbing robot can be estimated by using the center position, rotation angle, and identification ID of the Aruco tag, combined with three-dimensional acceleration and three-dimensional angular velocity.

[0065] Preferably, during the process of controlling the gimbal movement according to the degree of deviation, a PID feedback control algorithm is used to ensure that the Aruco marker is located at the center of the visual camera's field of view.

[0066] When using the PID feedback control algorithm, the desired value can be set as the position of the Aruco marker as the position of the center pixel. The coordinates of the Aruco marker in the coordinate system of the center pixel are compared with the position of the center pixel. The comparison result is fed back to the ground computer, and the ground computer performs adjustment control.

[0067] Preferably, the coordinates of the Aruco marker in the body coordinate system do not change with the movement of the wall-climbing robot.

[0068] In this embodiment, taking advantage of the invariance of the Aruco marker's coordinates in the body coordinate system, the coordinates in the camera coordinate system are obtained by calculating the coordinate transformation. Then, a further coordinate transformation is performed to transform the coordinates of the Aruco marker in the camera coordinate system to the coordinates in the ground coordinate system, thereby obtaining the three-dimensional spatial coordinates of the Aruco marker.

[0069] The above methods combine the advantages of visual recognition technology and laser ranging technology, and are not limited by GPS, inertial navigation or odometer. They are particularly suitable for robot positioning in enclosed environments, steel structure environments and other areas where GPS signals cannot cover. They have the characteristics of strong real-time performance, high accuracy and strong anti-interference ability, which greatly improves the efficiency and safety of wall-climbing robots in actual operation and are applicable to a variety of application scenarios in industrial sites.

[0070] Example 2

[0071] Based on the same principle as the aforementioned method, a spatial positioning device 100 for a wall-climbing robot based on visual recognition is also proposed, such as... Figure 3 As shown, it includes:

[0072] The Aruco marking module 110 is set to set Aruco markings of preset size and number on the body of the wall-climbing robot, and to construct a body coordinate system based on the body of the wall-climbing robot to determine the marking coordinates of the Aruco markings in the body coordinate system.

[0073] The capture module 120 is used to capture the Aruco marker by setting a visual camera on the gimbal, and to read the real-time pitch angle and real-time yaw angle of the gimbal during the capture process.

[0074] The marker coordinate transformation module 130 is used to construct a camera coordinate system based on the visual camera, analyze the real-time pitch angle and real-time yaw angle of the gimbal according to the PnP algorithm, and determine the coordinates of the Aruco marker in the camera coordinate system.

[0075] The three-dimensional spatial coordinate calculation module 140 is used to measure the straight-line distance between the gimbal and the wall-climbing robot by setting a laser rangefinder on the gimbal, and to analyze the coordinates of the Aruco marker in the camera coordinate system based on the straight-line distance, the real-time pitch angle and the real-time yaw angle, and to calculate the three-dimensional spatial coordinates of the wall-climbing robot in the ground coordinate system with the ground as the reference.

[0076] Example 3

[0077] Furthermore, an electronic device is proposed, comprising:

[0078] processor;

[0079] Memory used to store processor-executable instructions;

[0080] The processor is configured to implement the vision-based spatial positioning method for wall-climbing robots described in Embodiment 1 when executing the executable instructions.

[0081] It should be noted that in this invention, the use of terms such as "first," "second," and "a" is for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of that feature. In the description of this invention, "a plurality of" means at least two, such as two, three, etc., unless otherwise explicitly specified. The terms "connection," "fixed," etc., should be interpreted broadly. For example, "fixed" can mean a fixed connection, a detachable connection, or an integral part; it can mean a mechanical connection or an electrical connection; it can mean a direct connection or an indirect connection through an intermediate medium; it can mean the internal communication of two elements or the interaction between two elements, unless otherwise explicitly specified. Those skilled in the art can understand the specific meaning of the above terms in this invention according to the specific circumstances.

[0082] Furthermore, the technical solutions of the various embodiments of the present invention can be combined with each other, but only if they are feasible for those skilled in the art. If the combination of technical solutions is contradictory or cannot be implemented, it should be considered that such combination of technical solutions does not exist and is not within the scope of protection claimed by the present invention.

[0083] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

Claims

1. A spatial localization method for a wall-climbing robot based on visual recognition, characterized in that, include: Set Aruco markers of preset size and number on the body of the wall-climbing robot, and construct a body coordinate system based on the body of the wall-climbing robot to determine the marker coordinates of the Aruco markers in the body coordinate system; The Aruco marker was captured by a visual camera mounted on the gimbal, and the real-time pitch and yaw angles of the gimbal were read during the capture process. The method also includes capturing the Aruco marker by setting a visual camera on the gimbal, and further includes: The position of the wall-climbing robot is initially detected using the YOLO algorithm, and a captured image is generated. Using the coordinates of the center pixel of the captured image as a reference, the degree of deviation is calculated in the captured image based on the coordinates of the Aruco marker in the coordinate system where the center pixel is located, and the gimbal movement is controlled according to the degree of deviation. The generation of captured images refers to the real-time detection of the position information of the wall-climbing robot using the YOLO algorithm, the real-time capture of Aruco marker images on the wall-climbing robot using a vision camera, and the continuous tracking of the wall-climbing robot by the vision camera with the coordinates of the center pixel of the captured image as a reference, so as to keep it at the center of the viewpoint. Also includes: Extract the pose information of the Aruco marker from the captured image; Obtain the IMU information of the wall-climbing robot; The pose information includes the center position, rotation angle, and identification ID of the Aruco marker; The IMU information includes the three-dimensional acceleration and three-dimensional angular velocity of the wall-climbing robot; Using the visual camera as a reference, a camera coordinate system is constructed. The real-time pitch angle and real-time yaw angle of the gimbal are analyzed according to the PnP algorithm to determine the coordinates of the Aruco marker in the camera coordinate system. Using the visual camera as a reference, a camera coordinate system is constructed. Based on the PnP algorithm, the real-time pitch and yaw angles of the gimbal are analyzed to determine the coordinates of the Aruco marker in the camera coordinate system, including: Obtain the initial pitch angle and initial yaw angle of the gimbal in the camera coordinate system; The initial pitch angle, initial yaw angle, real-time pitch angle, and real-time yaw angle are analyzed using PnP to obtain the rotation matrix and displacement vector; Based on the rotation matrix and the displacement vector, coordinate transformation calculations are performed on the marker coordinates to determine the coordinates of the Aruco marker in the camera coordinate system; By measuring the straight-line distance between the gimbal and the wall-climbing robot using a laser rangefinder, and analyzing the coordinates of the Aruco marker in the camera coordinate system based on the straight-line distance, the real-time pitch angle, and the real-time yaw angle, the three-dimensional spatial coordinates of the wall-climbing robot in the ground coordinate system with the ground as the reference are calculated.

2. The spatial localization method for a wall-climbing robot based on visual recognition according to claim 1, characterized in that, By measuring the straight-line distance between the gimbal and the wall-climbing robot using a laser rangefinder mounted on the gimbal, and analyzing the coordinates of the Aruco marker in the camera coordinate system based on the straight-line distance, the real-time pitch angle, and the real-time yaw angle, the three-dimensional spatial coordinates of the wall-climbing robot in a ground coordinate system with the ground as the reference are calculated, including: The spatial coordinates of the visual camera in the ground coordinate system are calculated based on the straight-line distance, the real-time pitch angle, and the real-time yaw angle. The spatial coordinates of the vision camera and the coordinates of the Aruco marker in the camera coordinate system are transformed to obtain the three-dimensional spatial coordinates of the wall-climbing robot in the ground coordinate system with the ground as the reference.

3. The spatial localization method for a wall-climbing robot based on visual recognition according to claim 1, characterized in that, The coordinates of the Aruco marker in the coordinate system of the center pixel include: The pose of the wall-climbing robot is estimated based on the pose information and the IMU information, and the coordinates of the Aruco marker in the coordinate system of the center pixel are determined.

4. The spatial positioning method for a wall-climbing robot based on visual recognition according to claim 1, characterized in that, Also includes: During the process of controlling the gimbal movement according to the degree of deviation, a PID feedback control algorithm is used to keep the Aruco marker at the center of the visual camera's field of view.

5. The spatial localization method for a wall-climbing robot based on visual recognition according to claim 1, characterized in that, The coordinates of the Aruco marker in the body coordinate system do not change as the wall-climbing robot moves.

6. A spatial positioning device for a wall-climbing robot based on visual recognition, characterized in that, include: The Aruco marking module is set up to set Aruco markings of preset size and number on the body of the wall-climbing robot, and to construct a body coordinate system based on the body of the wall-climbing robot to determine the marking coordinates of the Aruco markings in the body coordinate system. The capture module is used to capture the Aruco marker by setting a vision camera on the gimbal, and to read the real-time pitch angle and real-time yaw angle of the gimbal during the capture process. The capture of the Aruco marker by setting a vision camera on the gimbal also includes: initially detecting the position of the wall-climbing robot using the YOLO algorithm, generating a capture image, and calculating the deviation degree of the Aruco marker in the capture image based on the coordinates of the center pixel of the capture image, and controlling the movement of the gimbal based on the deviation degree. Specifically, generating the capture image refers to detecting and viewing the position information of the wall-climbing robot in real time using the YOLO algorithm, capturing the Aruco marker image on the wall-climbing robot in real time using the vision camera, and continuously tracking the wall-climbing robot and keeping it at the center of the viewpoint using the coordinates of the center pixel of the capture image as a reference. The module also includes: extracting the pose information of the Aruco marker from the capture image, and obtaining the IMU information of the wall-climbing robot. The pose information includes the center position, rotation angle, and identification ID of the Aruco marker, and the IMU information includes the three-dimensional acceleration and three-dimensional angular velocity of the wall-climbing robot. The marker coordinate transformation module is used to construct a camera coordinate system based on the visual camera, analyze the real-time pitch angle and real-time yaw angle of the gimbal using the PnP algorithm, and determine the coordinates of the Aruco marker in the camera coordinate system. The module includes: obtaining the initial pitch angle and initial yaw angle of the gimbal in the camera coordinate system; performing PnP analysis on the initial pitch angle, initial yaw angle, real-time pitch angle, and real-time yaw angle to obtain a rotation matrix and a displacement vector; and performing coordinate transformation calculations on the marker coordinates based on the rotation matrix and the displacement vector to determine the coordinates of the Aruco marker in the camera coordinate system. The module for calculating three-dimensional spatial coordinates is used to measure the straight-line distance between the gimbal and the wall-climbing robot by setting a laser rangefinder on the gimbal, and to analyze the coordinates of the Aruco marker in the camera coordinate system based on the straight-line distance, the real-time pitch angle and the real-time yaw angle, and to calculate the three-dimensional spatial coordinates of the wall-climbing robot in the ground coordinate system with the ground as the reference.

7. An electronic device, characterized in that, include: processor; Memory used to store processor-executable instructions; The processor is configured to implement the spatial positioning method for a wall-climbing robot based on vision recognition as described in any one of claims 1-5 when executing the executable instructions.

Images