Wheel-foot composite building inspection robot and positioning and grabbing method of mechanical arm

Abstract

The application discloses a wheel-foot combined building inspection robot, which comprises a trolley main module, a radar sensor module, a depth camera module and a star wheel structure, the radar sensor module and the depth camera module are fixed on the trolley main module, the star wheel structure comprises a plurality of star wheels and a motor, the star wheels are respectively arranged on left and right sides of the trolley main module, the star wheel comprises a star wheel frame and a plurality of wheel plates, the star wheel frame is rotationally connected with the trolley main module and is driven to rotate by the motor, the star wheel frame is composed of a plurality of rib plates which expand outward, one wheel plate is connected to the outer side end of each rib plate, the plurality of wheel plates are distributed in a ring shape, and there is a spacing between adjacent wheel plates. Through the star wheel structure, the spacing between the wheel plates can be utilized to avoid slipping and the like, and the spacing between the wheel plates can be utilized to enhance the holding force of the star wheel, so that the motion limitation caused by the slope can be overcome.

Description

Technical Field

[0001] This invention relates to the field of robotics, specifically to a wheel-foot hybrid building inspection robot and a positioning and grasping method for its robotic arm. Background Technology

[0002] Inspection robots are comprehensive systems integrating environmental perception, target decision-making, and visual judgment. As a major branch of mobile robot applications, they embody research achievements from multiple disciplines, including signal processing, computer science, automation control engineering, and artificial intelligence. With the development of science and technology, inspection robots are widely used in hospitals, construction sites, power grid sites, and other areas, serving fields such as medicine, architecture, and electrical engineering. They are also frequently used in major national security areas such as wind power safety, urban safety, and national defense security. Compared to manual inspection, inspection robots have advantages such as anti-interference and stability, and are often used to inspect dangerous and harsh environments. In complex environments, the requirements for the grasping and positioning algorithms and accuracy of the robotic arms of construction inspection robots are also very high. Through the cooperation of inspection robots and robotic arms, machines can complete corresponding inspection tasks, replacing human work, ensuring human safety, and possessing significant economic benefits and broad market prospects.

[0003] However, under the current development situation, ordinary wheeled inspection robots have the disadvantage of poor resistance to environmental interference. They cannot adapt well to the environment of construction sites with flying sand and stones. Their movement trajectory is affected by the unevenness of the road, which cannot meet the inspection needs of construction sites with their changing environments. In complex inspection environments, collisions caused by the movement of inspection robots, as well as environmental influences, all cause certain displacement deviations in the movement of inspection robots, thus affecting the accuracy and efficiency of inspection robots when used.

[0004] Therefore, there is an urgent need for a wheel-foot hybrid building inspection robot to solve the problem that existing wheeled inspection robots are affected by uneven roads, which affects their accuracy and efficiency. Summary of the Invention

[0005] This invention addresses the shortcomings of existing technologies by providing a wheel-foot hybrid building inspection robot and a positioning and grasping method for its robotic arm. This solves the problem that existing wheeled inspection robots suffer from reduced accuracy and efficiency due to uneven roads.

[0006] To achieve the above objectives, the present invention adopts the following technical solution:

[0007] A wheel-foot hybrid building inspection robot includes a vehicle main module, a radar sensor module, a depth camera module, and a star wheel structure. The radar sensor module and the depth camera module are connected to the vehicle main module. The star wheel structure includes several star-shaped wheels and a motor. The star-shaped wheels are respectively installed on the left and right sides of the vehicle main module. Each star-shaped wheel includes a star-shaped wheel frame and several wheel plates. The star-shaped wheel frame is rotatably connected to the vehicle main module and is driven to rotate by the motor. The star-shaped wheel frame is composed of several outwardly extending ribs. Each outer end of the rib is connected to a wheel plate. The wheel plates are arranged in a ring, and there is a gap between adjacent wheel plates.

[0008] To optimize the above technical solution, the specific measures also include:

[0009] Furthermore, the star-shaped wheel frame includes rib plate one, rib plate two, rib plate three, rib plate four, and rib plate five. The wheel plate is an inwardly curved arc and includes wheel plate one, wheel plate two, wheel plate three, wheel plate four, and wheel plate five. The star-shaped wheel frame also includes an axle, which is used for rotatably connecting with the main body module of the vehicle. Ribs plate one, rib plate two, rib plate three, rib plate four, and rib plate five are equidistantly and radially connected to the outer wall of the axle. The outer ends of rib plate one, rib plate two, rib plate three, rib plate four, and rib plate five are respectively fixedly connected to the center of the inner side of wheel plate one, wheel plate two, wheel plate three, wheel plate four, and wheel plate five.

[0010] Furthermore, a rubber pad is installed on the outer surface of the wheel plate.

[0011] Furthermore, the main body module of the vehicle includes a vehicle body, which includes a main frame, a front front frame, a rear front frame, a front end, a rear end, a right side panel, a left side panel, a right front wheel axle, a right rear wheel axle, a left rear wheel axle, a left front wheel axle, a power module box, and a radar sensor mounting module. The front end of the main frame is the front end, where the front front frame is installed, and the rear end is the rear end, where the rear front frame is installed. The left side panel is installed on the left side of the main frame, and the left front wheel axle and left rear wheel axle are installed on the front and rear sides of the left side panel, respectively. The right side panel is installed on the right side, and the right front wheel axle and right rear wheel axle are installed on the front and rear sides of the right side panel, respectively. The radar sensor mounting module is installed on the upper end of the main frame for installing the radar sensor module. The radar sensor mounting module is also equipped with a power module box for installing the power module.

[0012] Furthermore, the radar sensor module includes a lidar, a fixed column, a rotating strip, and a drive motor. The lidar has a first circular hole at its top, and two circular holes at its outer ring. The sidewall of the lidar has two through holes, and a laser emission hole is located between the two through holes. The drive motor is connected to the main body module of the vehicle. The output shaft of the drive motor is connected to the fixed column. The fixed column and the bottom of the lidar are fitted with a rotating strip. The drive motor drives the fixed column to rotate, and the fixed column drives the lidar to rotate through the rotating strip.

[0013] Furthermore, the depth camera module includes a depth camera, which is equipped with an infrared laser emitter, an RGB camera, and an infrared camera. The front end of the depth camera is a mirror, and the infrared laser emitter, RGB camera, and infrared camera are all mounted on the mirror. The lower end of the depth camera is equipped with a left camera bracket and a right camera bracket, and the depth camera is connected and fixed to the main body module of the vehicle through the left camera bracket and the right camera bracket.

[0014] Furthermore, the left camera bracket includes a left camera bracket base support, a left camera bracket pivot plate, and a left camera bracket base plate, and the right camera bracket includes a right camera bracket base support, a right camera bracket pivot plate, and a right camera bracket base plate. The left camera bracket base plate and the right camera bracket base plate are installed at the lower end of the depth camera. The left camera bracket base support and the right camera bracket base support are correspondingly installed on the left camera bracket base plate and the right camera bracket base plate. The left camera bracket pivot plate is rotatably mounted on the left camera bracket base plate via the left camera bracket base support, and the right camera bracket base support is rotatably mounted on the right camera bracket base plate via the right camera bracket base support.

[0015] Furthermore, a positioning and grasping method for the robotic arm of a wheel-footed composite building inspection robot is provided.

[0016] The main body module of the vehicle is also equipped with a robotic arm, and a depth camera module is installed at the end of the robotic arm, parallel to the gripper of the robotic arm; the positioning and grasping method of the robotic arm includes the following steps:

[0017] Using deep learning technology, the information collected by the camera of the depth camera module is used to identify the object to be grasped, obtain the coordinates and depth information of the object in the image, establish the base coordinate system, object coordinate system and camera coordinate system, and obtain the coordinate transformation relationship between the base coordinate system, object coordinate system and camera coordinate system.

[0018] By calibrating the depth camera module, the camera's intrinsic and extrinsic parameters, as well as distortion coefficients, are obtained, thus yielding the object's three-dimensional coordinates relative to the camera, i.e., the object's relative pose. Then, using the eye-to-hand calibration model, the relative pose of the camera to the robotic arm's end effector is calculated. Furthermore, the robotic arm's built-in feedback mechanism provides feedback on the end effector's relative pose to the base coordinate system. Based on these four relative pose relationships, the three-dimensional coordinates of the object to be grasped relative to the robotic arm's base coordinate system are determined using the robot coordinate transformation equation. This three-dimensional coordinate system is then converted into a matrix and transmitted to the robotic arm controller, enabling the robotic arm to perform the object positioning and grasping task.

[0019] Furthermore, when solving for the 3D coordinates of the object to be captured relative to the camera, i.e., the 3D coordinates of the object in the camera coordinate system, the x-coordinates of the object in the image are obtained using OpenCV image processing technology. r y r , z r The depth h of the object to be captured is obtained through the camera. r The conversion ratio between image coordinates and actual coordinates is actual distance * object depth / pixel value, where the pixel value is b, and the conversion ratio is α. x α y α z ;

[0020] The magnification factor along the X-axis is:

[0021] The magnification factor along the Y axis is:

[0022] The Z-axis magnification factor is

[0023] Based on the object coordinate system and the camera coordinate system, the coordinate transformation relationship of the object's displacement relative to the camera is derived as follows:

[0024] When the robotic arm is directly facing the object to be grasped, the positional relationship between the object and the robotic arm base remains unchanged. The rotation matrix obtained at this time is the orientation rotation matrix A. The coordinate transformation relationship between the object and the camera is as follows:

[0025] Using the hand-eye calibration model, the rotation matrix M of the camera relative to the end effector is obtained based on the camera coordinate system and the end effector coordinate system. C→H Since the positional relationship between the camera and the end effector is relatively fixed, the rotation matrix M... C→H It is a fixed value;

[0026] Furthermore, based on the pose feedback device of the end effector built into the robotic arm, the relative pose relationship of the end effector with respect to the base coordinate system can be fed back. A 1*4 matrix M' = [x] is obtained through function calls. m y m z m θ],

[0027] Based on the coordinate system of the end effector and the base coordinate system, the displacement coordinate transformation relationship of the end effector relative to the base coordinate of the robotic arm is obtained as follows:

[0028] Where, x m It represents the magnitude of the end effector's displacement along the x-axis in the base coordinate system, i.e., horizontally to the right, and y-axis... m It represents the magnitude of the end effector's displacement along the y-axis in the base coordinate system, i.e., horizontally forward, and z-axis... m θ represents the displacement of the end effector along the z-axis in the base coordinate system, i.e., vertically upward. θ is the rotation angle of the end effector relative to the preset rotation zero point. Since the robotic arm rotates around the z-axis, the rotation matrix can be obtained.

[0029] Transform matrix M' into

[0030] In summary, the target rotation matrix is ​​M = M H→B M C→H M O→C The target rotation matrix is ​​communicated with Dobot through the ROS robot operating system and transmitted to the robotic arm to perform the grasping task.

[0031] Furthermore, the robotic arm is a Dobot robotic arm.

[0032] The beneficial effects of this invention are:

[0033] This invention, through the design of a star-wheel structure and the spacing between the wheel plates, enables the inspection robot to overcome the drawbacks of complex terrain during inspection tasks. It avoids slippage and other issues common with conventional wheeled robots on challenging surfaces, preventing changes in direction. Furthermore, when inspecting steep inclines, the star-wheel structure enhances the grip of the star-shaped wheels by increasing the spacing between the wheel plates, overcoming movement limitations caused by the slope and allowing for gentler inclines, thus broadening the robot's operational capabilities. The ring structure formed by several wheel plates increases the contact area, allowing the wheels to grip the ground better. This is particularly important for inspection vehicles or vehicles navigating complex road conditions, as the star-shaped wheels provide better traction and handling. Additionally, the multiple support points of the star-shaped wheels provide better shock absorption, reducing bumps and vibrations during vehicle operation and preventing deformation or displacement of sensors and other components on the vehicle.

[0034] This invention decomposes the actual three-dimensional coordinates of an object into three-dimensional spatial coordinates and the rotation angle of a robotic arm. Based on the rotation matrix formula and determining which axis the object rotates around, it converts the result into a corresponding rotation matrix. This method is more modular, simplifies the calculation process, and is more conducive to obtaining the required three-dimensional coordinates of the object, thereby controlling the robotic arm to perform recognition and grasping tasks. This invention uses deep learning to detect the object's position and calculate its pose to obtain its actual coordinates, then controls the robotic arm to grasp it. The process is simple, easy to implement, cost-effective, and has high positioning accuracy. Attached Figure Description

[0035] Figure 1 This is a schematic diagram of the overall structure of a wheel-foot composite building inspection robot proposed in this invention;

[0036] Figure 2 This is a schematic diagram of the main body module of a wheel-foot composite building inspection robot proposed in this invention;

[0037] Figure 3 This is a schematic diagram of the radar sensor module of a wheel-foot composite building inspection robot proposed in this invention;

[0038] Figure 4 This is a schematic diagram of the depth camera module of a wheel-foot hybrid building inspection robot proposed in this invention;

[0039] Figure 5 This is a schematic diagram of the bottom structure of the depth camera module of a wheel-foot composite building inspection robot proposed in this invention;

[0040] Figure 6This is a schematic diagram of the star wheel structure of a wheel-foot composite building inspection robot proposed in this invention;

[0041] Figure 7 This is a schematic diagram of the positioning method for the robotic arm of a wheel-foot composite building inspection robot proposed in this invention;

[0042] Figure 8 This is a schematic diagram of the positioning method for the robotic arm of a wheel-foot composite building inspection robot proposed in this invention.

[0043] Figure 9 This is a schematic diagram of the installation of the robotic arm of a wheel-foot composite building inspection robot proposed in this invention.

[0044] Reference numerals in the attached diagram: 1 is the main body module of the vehicle, 101 is the front frame, 102 is the rear frame, 103 is the front of the vehicle, 104 is the rear of the vehicle, 105 is the right side panel, 106 is the left side panel, 107 is the right front wheel axle, 108 is the right rear wheel axle, 109 is the left rear wheel axle, 110 is the left front wheel axle, 111 is the power module box, and 112 is the radar sensor mounting module.

[0045] 2 is the radar sensor module, 201 is the first circular hole, 202 is the second circular hole, 203 is the third circular hole, 204 is the laser emission hole, 205 is the first through hole, 206 is the second through hole, 207 is the fixed column, 208 is the rotating strip, and 209 is the drive motor.

[0046] 3 is the depth camera module, 301 is the infrared laser emitter, 302 is the RGB camera, 303 is the infrared camera, 304 is the mirror, 305 is the left camera bracket, 30501 is the bottom support of the left camera bracket, 30502 is the pivot plate of the left camera bracket, 30503 is the base plate of the left camera bracket, 306 is the right camera bracket, 30601 is the bottom support of the right camera bracket, 30602 is the pivot plate of the right camera bracket, and 30603 is the base plate of the right camera bracket.

[0047] 4 is a star wheel structure, 401 is wheel plate one, 402 is wheel plate two, 403 is wheel plate three, 404 is wheel plate four, 405 is wheel plate five, 406 is rib plate one, 407 is rib plate two, 408 is rib plate three, 409 is rib plate four, 410 is rib plate five, and 411 is axle. Detailed Implementation

[0048] The invention will now be described in further detail with reference to the accompanying drawings.

[0049] As attached Figure 1As shown, an embodiment of the present invention discloses a wheel-foot composite building inspection robot, comprising a vehicle main module 1, a radar sensor module 2, a depth camera module 3, and a star wheel structure 4. The radar sensor module 2 and the depth camera module 3 are connected to the vehicle main module 1. The star wheel structure 4 includes several star-shaped wheels and a motor. The several star-shaped wheels are respectively installed on the left and right sides of the vehicle main module 1. The star-shaped wheel includes a star-shaped wheel frame and several wheel plates. The star-shaped wheel frame is rotatably connected to the vehicle main module 1 and is driven to rotate by the motor. The star-shaped wheel frame is composed of several outwardly extending ribs. A wheel plate is connected to the outer end of each rib. The several wheel plates are arranged in a ring, and there is a gap between adjacent wheel plates.

[0050] Therefore, by using the star wheel structure 4 and the spacing between the wheel plates, the inspection robot can overcome the drawbacks caused by the complexity of the ground during inspection tasks. This avoids slippage and other issues that can occur with ordinary wheeled robots on complex terrain, preventing changes in direction of movement. Furthermore, when performing inspection tasks on steep slopes, the star wheel inspection robot can enhance the gripping force of the star-shaped wheels through the spacing between the wheel plates, overcoming the movement limitations imposed by the slope and enabling it to perform tasks on gentler inclines, thus broadening the robot's movement capabilities. The ring structure formed by several wheel plates increases the contact area, allowing the wheels to grip the ground better. This is particularly important for inspection vehicles or vehicles that need to navigate complex road conditions, as the gripping performance of the star-shaped wheels provides better traction and handling. Additionally, the multiple support points of the star-shaped wheels provide better shock absorption, reducing bumps and vibrations during vehicle operation and preventing deformation or displacement of sensors and other devices on the vehicle body.

[0051] In this embodiment, the main vehicle module 1 is used to realize the movement of the main vehicle, and the star wheel structure 4 is used as the driving wheel structure of the vehicle. The radar sensor module 2 is used to measure the distance and transmit the distance information of the inspection site to the main vehicle module 1. The depth camera module 3 is used to identify specific objects in the inspection site and return the corresponding three-dimensional coordinates to the main vehicle module 1, so that the main vehicle module 1 can process the three-dimensional coordinate information. The star wheel structure 4 serves as the driving device of the vehicle and is suitable for inspection scenarios with complex and varied ground conditions.

[0052] For details, see attached. Figure 6As shown, in one embodiment, the ribs of the aforementioned star-shaped wheel frame may include rib 1 406, rib 2 407, rib 3 408, rib 409, and rib 5 410. The wheel plates are inwardly curved arcs and include wheel 1 401, wheel 2 402, wheel 3 403, wheel 404, and wheel 5 405. The star-shaped wheel frame also includes an axle 411, which is used for rotatable connection with the main body module 1 of the vehicle. The outer wall of the axle 411 is radially connected with ribs 406, 407, 408, 409, and 410 at equal intervals. The outer ends of ribs 406, 407, 408, 409, and 410 are fixedly connected to the centers of the inner sides of wheel plates 401, 402, 403, 404, and 405, respectively. In this embodiment, a star-wheel structure is used to modify the vehicle body. The number of teeth in the star-wheel should not be too many, as too many teeth will lead to an increase in the slippage rate, while too few teeth will lead to poor wheel stability. In the application of inspection robots, the mechanical structure that drives the machine is particularly important. This solution is suitable for inspection sites with complex and changing environments, and the modification of its mechanical structure improves the stability of the vehicle body.

[0053] The outer surface of the wheel plate is fitted with a rubber pad, which increases the buffering force and stability of the device.

[0054] As attached Figure 2 As shown, in another embodiment, the main body module 1 of the vehicle includes a vehicle body, a drive unit, and an electrical control unit. The vehicle body, drive unit, and electrical control unit work together to drive the inspection robot to move. The drive unit can be the aforementioned motor. The vehicle body includes a main frame, a front frame 101, a rear frame 102, a front end 103, a rear end 104, a right side panel 105, a left side panel 106, a right front axle 107, a right rear axle 108, a left rear axle 109, a left front axle 110, a power module box 111, and a radar sensor mounting module 112. The front end of the main frame is the front end 103. The vehicle is equipped with a front frame 101 at three locations, a rear frame 104 at the rear, and a rear frame 102 at the rear frame 104. A left side panel 106 is mounted on the left side of the main frame, with a left front wheel axle 110 and a left rear wheel axle 109 mounted on its front and rear sides respectively. A right side panel 105 is mounted on the right side, with a right front wheel axle 107 and a right rear wheel axle 108 mounted on its front and rear sides respectively. A radar sensor mounting module 112 is mounted on the upper part of the main frame for mounting a radar sensor module 2. The radar sensor mounting module 112 also has a power module box 111 for mounting the power module. The depth camera module 3, radar sensor module 2, and main control module can be integrated and fixed onto the radar sensor mounting module 112.

[0055] In this embodiment, screws can be used for fixing. The vehicle body is made of aluminum alloy to reinforce it, improving the robot's durability in harsh environments and increasing its stability. A power bank with a capacity of 10000mAh or more is used to power the host computer, ensuring power stability and battery life.

[0056] As attached Figure 3 As shown, in another embodiment, the radar sensor module 2 includes a lidar, a fixed column 207, a rotating strip 208, and a drive motor 209. The lidar has a first circular hole 201 on its top, and two circular holes 202 and 203 on its outer ring. The sidewall of the lidar has two through holes 205 and 206, with a laser emission hole 204 between them. The drive motor 209 is connected to the main body module 1 of the vehicle. The output shaft of the drive motor 209 is connected to the fixed column 207. The rotating strip 208 is fitted onto the bottom of the fixed column 207 and the lidar. The drive motor 209 drives the fixed column 207 to rotate, and the fixed column 207 drives the lidar to rotate via the rotating strip 208. In this embodiment, the drive motor 209 and other equipment can be installed in the aforementioned radar sensor mounting module 112. In this embodiment, any radar can be selected; however, to adapt to the size of the inspection site, a high-precision radar should be chosen.

[0057] As attached Figure 4 As shown, in another embodiment, the depth camera module 3 includes a depth camera, which is equipped with three visual elements: an infrared laser emitter 301, an RGB camera 302, and an infrared camera 303. The front end of the depth camera is a mirror 304, and the infrared laser emitter 301, the RGB camera 302, and the infrared camera 303 are all mounted on the mirror 304. The lower end of the depth camera is equipped with a left camera bracket 305 and a right camera bracket 306, respectively. The depth camera is connected and fixed to the main body module 1 of the vehicle through the left camera bracket 305 and the right camera bracket 306.

[0058] Among them, as attached Figure 5As shown, the left camera bracket 305 includes a left camera bracket base support 30501, a left camera bracket pivot plate 30502, and a left camera bracket base plate 30503. The right camera bracket 306 includes a right camera bracket base support 30601, a right camera bracket pivot plate 30602, and a right camera bracket base plate 30603. The left camera bracket base plate 30503 and the right camera bracket base plate 30603 are installed at the lower end of the depth camera. The left camera bracket base support 30501 and the right camera bracket base support 30601 are correspondingly installed on the left camera bracket base plate 30501 and the right camera bracket base plate 30603, respectively. The left camera bracket pivot plate 30502 is rotatably mounted on the left camera bracket base plate 30503 via the left camera bracket base support 30501, and the right camera bracket base support 30601 is rotatably mounted on the right camera bracket base plate 30603 via the right camera bracket base support 30601. In this embodiment, the pivots of the left camera bracket pivot plate 30502 and the right camera bracket pivot plate 30602 respectively pass through the left camera bracket base support 30501 and the right camera bracket base support 30601, and are threaded, allowing for tightening or loosening with nuts. This enables the rotation adjustment and fixation of the left camera bracket pivot plate 30502 and the right camera bracket pivot plate 30602. Using pivot plates that can rotate around an axis for fixing the depth camera allows the depth camera to have a steering function, making it more sensitive.

[0059] As attached Figure 7 and attached Figure 8 As shown in the attached figure, a positioning and grasping method for the robotic arm of a wheel-footed composite building inspection robot is described. Figure 9 As shown, a robotic arm is also provided on the aforementioned main body module 1 of the vehicle. In this embodiment, the depth camera of the depth camera module 3 is installed at the end of the robotic arm and is parallel to the gripper of the robotic arm. The positioning and grasping method of the robotic arm includes the following steps:

[0060] Using deep learning technology, the information collected by the camera in depth camera module 3 is used to identify the object to be grasped, and to obtain the object's coordinates and depth information in the image. The camera in depth camera module 3 is equivalent to the depth camera mentioned above. A RealSense D435i depth camera can be used to obtain the object's depth information and establish a base coordinate system, an object coordinate system, and a camera coordinate system. The coordinate transformation relationship between the base coordinate system, the object coordinate system, and the camera coordinate system is also obtained. The host computer can be used for deep learning recognition. After the object is identified, the image coordinates and depth information of the object are calculated, and a base coordinate system is established based on the base of the robotic arm. The required host computer is a small and powerful Raspberry Pi 4B. Other main controllers can also be used. The main control board can be fixed and packaged by tightening screws and nuts.

[0061] By calibrating the depth camera module 3, the camera's intrinsic and extrinsic parameters, as well as distortion coefficients, are obtained, thus yielding the three-dimensional coordinates of the object to be grasped relative to the camera, i.e., the relative pose of the object to be grasped relative to the camera. Then, using the eye-to-hand calibration model of the hand-eye calibration model, the relative pose of the camera relative to the end effector of the robotic arm is calculated. Furthermore, the robotic arm's built-in feedback mechanism provides feedback on the relative pose of the end effector relative to the base coordinate system. Based on these four relative pose relationships, the three-dimensional coordinates of the object to be grasped relative to the base coordinate system of the robotic arm can be determined using the robot coordinate transformation equation. This three-dimensional coordinate is then converted into a matrix and transmitted to the robotic arm controller, driving the robotic arm to perform the object positioning and grasping task. Since the Dobot robotic arm's Z-axis is fixed and the end effector rotates around the Z-axis, the three-dimensional coordinates of the camera relative to the base coordinate system are solved by matrix multiplication, thus obtaining the three-dimensional coordinates of the object relative to the base coordinate system.

[0062] Specifically, when calculating the three-dimensional coordinates of the object to be captured relative to the camera—that is, the three-dimensional coordinates of the object in the camera coordinate system—OpenCV image processing technology is used to obtain the object's coordinates x in the image. r y r , z r The depth h of the object to be captured is obtained through a depth camera. r According to the principle of pinhole imaging, the conversion ratio between image coordinates and actual coordinates is actual distance * object depth / pixel value, where the pixel value is b. Similarly, let this conversion ratio be α. x α y α z ;

[0063] The magnification factor along the X-axis is:

[0064] The magnification factor along the Y axis is:

[0065] The Z-axis magnification factor is

[0066] Based on the object coordinate system and the camera coordinate system, the coordinate transformation relationship of the object's displacement relative to the camera is derived as follows:

[0067] When the robotic arm is directly facing the object to be grasped, the positional relationship between the object and the robotic arm base remains unchanged. The rotation matrix obtained at this time is the orientation rotation matrix A. The coordinate transformation relationship between the object and the camera is as follows:

[0068] Using the hand-eye calibration model, the rotation matrix M of the camera relative to the end effector is obtained based on the camera coordinate system and the end effector coordinate system. C→H Since the positional relationship between the camera and the end effector is relatively fixed, the rotation matrix M... C→H It is a fixed value;

[0069] Furthermore, based on the pose feedback device of the end effector built into the robotic arm, the relative pose relationship of the end effector with respect to the base coordinate system can be fed back. A 1*4 matrix M' = [x] is obtained through function calls. m y m z m [θ], by simplifying the required 4×4 rotation matrix to a 1×4 matrix, it is easier to implement different rotation strategies.

[0070] Based on the coordinate system of the end effector and the base coordinate system, the displacement coordinate transformation relationship of the end effector relative to the base coordinate of the robotic arm is obtained as follows:

[0071] Where, x m It represents the magnitude of the end effector's displacement along the x-axis in the base coordinate system, i.e., horizontally to the right, and y-axis... m It represents the magnitude of the end effector's displacement along the y-axis in the base coordinate system, i.e., horizontally forward, and z-axis... m θ represents the displacement of the end effector along the z-axis in the base coordinate system, i.e., vertically upward. θ is the rotation angle of the end effector relative to the preset rotation zero point. Since the robotic arm rotates around the z-axis, the rotation matrix can be obtained.

[0072] Transform the rotation matrix M' into

[0073] In summary, the target rotation matrix is ​​M = M H→B M C→H M O→C The target rotation matrix is ​​communicated with Dobot through the ROS robot operating system and transmitted to the robotic arm to perform the grasping task.

[0074] The robotic arm mentioned above is a Dobot robotic arm.

[0075] This invention decomposes the actual three-dimensional coordinates of an object into three-dimensional spatial coordinates and the rotation angle of the robotic arm. Based on the rotation matrix formula and determining which axis the object rotates around, it converts the result into a corresponding rotation matrix. This method is more modular, simplifies the calculation process, and is more conducive to obtaining the required three-dimensional coordinates of the object, thereby controlling the Dobot robotic arm to perform recognition and grasping tasks. This invention uses deep learning to detect the object's position and calculate its pose to obtain its actual coordinates, then controls the Dobot robotic arm to grasp it. The process is simple, easy to implement, cost-effective, and has high positioning accuracy.

[0076] This invention allows for system control via Raspberry Pi and STM32, utilizing LiDAR mapping technology to display complex and ever-changing inspection sites in map form. It also incorporates deep learning vision to identify specific objects within the inspection site, meeting the various requirements of an inspection robot. Furthermore, its modular design allows for good scalability and development potential.

[0077] This invention addresses the shortcomings of existing wheeled robots in terms of poor resistance to environmental interference. It employs a star-wheel structure design, transforming a conventional wheeled inspection robot into a wheel-foot hybrid inspection robot. This design minimizes the impact of obstacles in complex environments, stabilizing its movement trajectory and enabling precise completion of inspection tasks. The depth camera, LiDAR, and main control modules are integrated and fixed, facilitating disassembly and replacement and reducing maintenance costs. The Dobot robotic arm, combined with deep learning-based positioning and recognition, performs localization and grasping, improving the accuracy and efficiency of the robotic arm's positioning.

[0078] It should be noted that the terms such as "upper", "lower", "left", "right", "front", and "back" used in the invention are only for clarity of description and are not intended to limit the scope of the invention. Changes or adjustments to their relative relationships, without substantially altering the technical content, should also be considered within the scope of the invention.

[0079] The above are merely preferred embodiments of the present invention. The scope of protection of the present invention is not limited to the above embodiments. All technical solutions falling within the scope of the present invention's concept are within the scope of protection of the present invention. It should be noted that for those skilled in the art, any improvements and modifications made without departing from the principles of the present invention should be considered within the scope of protection of the present invention.

Claims

1. A wheel-foot hybrid building inspection robot, characterized in that: The system includes a car body module (1), a radar sensor module (2), a depth camera module (3), and a star wheel structure (4). The car body module (1) is connected to the radar sensor module (2) and the depth camera module (3). The star wheel structure (4) includes several star-shaped wheels and a motor. Several star-shaped wheels are respectively installed on the left and right sides of the car body module (1). The star-shaped wheel includes a star-shaped wheel frame and several wheel plates. The star-shaped wheel frame is rotatably connected to the car body module (1) and is driven to rotate by the motor. The star-shaped wheel frame is composed of several outwardly extending ribs. Each outer end of the rib is connected to a wheel plate. Several wheel plates are arranged in a ring, and there is a gap between adjacent wheel plates. The star-shaped wheel frame has ribs including rib one (406), rib two (407), rib three (408), rib four (409), and rib five (410). The wheel plates are inwardly curved arcs and include wheel plate one (401), wheel plate two (402), wheel plate three (403), wheel plate four (404), and wheel plate five (405). The star-shaped wheel frame also includes an axle (411), which is used for rotatable connection with the main body module (1) of the vehicle. The outer wall is connected radially with ribs 1 (406), rib 2 (407), rib 3 (408), rib 4 (409) and rib 5 (410) at equal intervals. The outer ends of ribs 1 (406), rib 2 (407), rib 3 (408), rib 4 (409) and rib 5 (410) are respectively fixedly connected to the center of the inner side of wheel plate 1 (401), wheel plate 2 (402), wheel plate 3 (403), wheel plate 4 (404) and wheel plate 5 (405). The depth camera module (3) includes a depth camera, which is equipped with an infrared laser emitter (301), an RGB camera (302) and an infrared camera (303). The front end of the depth camera is a mirror (304). The infrared laser emitter (301), the RGB camera (302) and the infrared camera (303) are all mounted on the mirror (304). The lower end of the depth camera is equipped with a left camera bracket (305) and a right camera bracket (306). The depth camera is connected and fixed to the main body module (1) of the car through the left camera bracket (305) and the right camera bracket (306). The left camera bracket (305) includes a left camera bracket base support (30501), a left camera bracket pivot plate (30502), and a left camera bracket base plate (30503). The right camera bracket (306) includes a right camera bracket base support (30601), a right camera bracket pivot plate (30602), and a right camera bracket base plate (30603). The left camera bracket base plate (30503) and the right camera bracket base plate (30603) are mounted on the lower end of the depth camera. The left camera bracket base support (30501) and the right camera bracket base support (30603) are respectively installed on the plate (30503) and the right camera bracket base plate (30603). The left camera bracket pivot plate (30502) is rotatably mounted on the left camera bracket base plate (30503) through the left camera bracket base support (30501), and the right camera bracket pivot plate (30602) is rotatably mounted on the right camera bracket base plate (30603) through the right camera bracket base support (30601).

2. The wheel-foot composite building inspection robot according to claim 1, characterized in that: A rubber pad is installed on the outer surface of the wheel plate.

3. The wheel-foot composite building inspection robot according to claim 1, characterized in that: The main body module (1) of the vehicle includes a vehicle body, which includes a main frame, a front front frame (101), a rear front frame (102), a front (103), a rear (104), a right side panel (105), a left side panel (106), a right front wheel axle (107), a right rear wheel axle (108), a left rear wheel axle (109), a left front wheel axle (110), a power module box (111), and a radar sensor mounting module (112). The front end of the main frame is the front (103), where the front front frame (101) is mounted, and the rear end is the rear (104). 4) A rear front frame (102) is installed at the main frame. A left side plate (106) is installed on the left side of the main frame. A left front wheel axle (110) and a left rear wheel axle (109) are installed on the front and rear sides of the left side plate (106) respectively. A right side plate (105) is installed on the right side. A right front wheel axle (107) and a right rear wheel axle (108) are installed on the front and rear sides of the right side plate (105) respectively. A radar sensor mounting module (112) is installed at the upper end of the main frame for installing a radar sensor module (2). A power module box (111) is also provided on the radar sensor mounting module (112) for installing a power module.

4. The wheel-foot composite building inspection robot according to claim 1, characterized in that: The radar sensor module (2) includes a lidar, a fixed column (207), a rotating strip (208), and a drive motor (209). The lidar has a circular hole 1 (201) on its top and a circular hole 2 (202) and a circular hole 3 (203) on its outer ring. The lidar has a through hole 1 (205) and a through hole 2 (206) on its side wall. A laser emission hole (204) is provided between the through hole 1 (205) and the through hole 2 (206). The drive motor (209) is connected to the main body module (1) of the vehicle. The output shaft of the drive motor (209) is connected to the fixed column (207). The fixed column (207) and the bottom of the lidar are fitted with a rotating strip (208). The drive motor (209) is used to drive the fixed column (207) to rotate. The fixed column (207) drives the lidar to rotate through the rotating strip (208).

5. The positioning and grasping method of the robotic arm of a wheel-foot composite building inspection robot according to claim 1, characterized in that, The main body module (1) of the vehicle is also equipped with a robotic arm, and the depth camera module (3) is installed at the end of the robotic arm and is parallel to the gripper of the robotic arm; the positioning and grasping method of the robotic arm includes the following steps: Using deep learning technology, the information collected by the camera of the depth camera module (3) is used to identify the object to be grasped, obtain the coordinates and depth information of the object in the image, establish the base coordinate system, the object coordinate system and the camera coordinate system, and obtain the coordinate transformation relationship between the base coordinate system, the object coordinate system and the camera coordinate system. By calibrating the depth camera module (3), the intrinsic parameters, extrinsic parameters, and distortion coefficients of the camera are obtained, thereby obtaining the three-dimensional coordinates of the object relative to the camera, that is, the relative pose relationship of the object relative to the camera; then, by using the eye of the hand-eye calibration model to calibrate the model on the hand, the relative pose relationship of the camera relative to the end effector of the robotic arm is calculated; then, according to the feedback device built into the robotic arm, the relative pose relationship of the end effector relative to the base coordinate system can be fed back; based on the relative pose relationship of the above four units, the three-dimensional coordinates of the object to be grasped relative to the base coordinate system of the robotic arm can be determined by the robot coordinate transformation equation; the three-dimensional coordinates are converted into a matrix and transmitted to the robotic arm controller, thereby driving the robotic arm to perform the task of positioning and grasping the object.

6. The positioning and grasping method of the robotic arm of a wheel-foot composite building inspection robot according to claim 5, characterized in that: When determining the 3D coordinates of the object to be captured relative to the camera, i.e., the 3D coordinates of the object in the camera coordinate system, OpenCV image processing technology is used to obtain the object's coordinates in the image. , , The depth of the object to be captured is obtained through the camera. The conversion ratio between image coordinates and actual coordinates is equal to the actual distance. Object depth / pixel value, pixel value Let the conversion ratio be... , , ; The magnification factor along the X-axis is: The magnification factor along the Y axis is: The Z-axis magnification factor is then... Based on the object coordinate system and the camera coordinate system, the coordinate transformation relationship of the object's displacement relative to the camera is derived as follows: , When the robotic arm is directly facing the object to be grasped, the positional relationship between the object and the robotic arm base remains unchanged. The rotation matrix obtained at this time is the orientation rotation matrix A. Then the coordinate transformation relationship between the object and the camera is: , Using the hand-eye calibration model, the rotation matrix of the camera relative to the end effector is obtained based on the camera coordinate system and the end effector coordinate system. Since the positional relationship between the camera and the end effector is relatively fixed, the rotation matrix... It is a fixed value; Furthermore, based on the pose feedback device of the end effector built into the robotic arm, the relative pose relationship of the end effector with respect to the base coordinate system can be fed back, and 1 can be obtained through function calls. 4-matrix , Based on the coordinate system of the end effector and the base coordinate system, the displacement coordinate transformation relationship of the end effector relative to the base coordinate of the robotic arm is obtained as follows: , in, The end effector in the base coordinate system The magnitude of the axis displacement, i.e., horizontally to the right. The end effector in the base coordinate system The magnitude of the axis displacement, i.e., horizontally forward. The end effector in the base coordinate system The magnitude of the axis displacement, i.e., vertically upward. The rotation angle of the end effector relative to the preset rotation zero point, due to the robot arm's rotation... Rotating the axis yields the rotation matrix. = , matrix Transform into ; In summary, the target rotation matrix is: The target rotation matrix is ​​communicated with Dobot through the ROS robot operating system and transmitted to the robotic arm to perform the grasping task.

7. The positioning and grasping method of the robotic arm of a wheel-foot composite building inspection robot according to claim 6, characterized in that: The robotic arm used is a Dobot robotic arm.

Images