Four-foot line patrol robot based on gyroscope and visual data fusion

Abstract

The invention discloses a four-foot line patrol robot based on gyroscope and visual data fusion, and relates to the technical field of robots, the two ends of a trunk support are provided with eight motor mounting positions, each motor mounting position is provided with a driving motor, and the driving motor is connected with, drives and controls one advancing leg through a matched coupler; the four sets of advancing legs are assembled at the corresponding positions of the trunk support respectively, the main control device is connected with the eight driving motors through signal lines respectively, and four-foot advancing control over the robot is achieved. Compared with the prior art, the method has the advantages that the advantage that the visual sensor provides rich information in a normal environment and the stability that the IMU provides pose estimation in a severe environment are combined, and the line patrol precision and stability of the robot in various environments are effectively improved; through PID closed-loop control and cascade PID closed-loop control, the step length and the leg length difference value of the advancing legs can be accurately controlled, and stable operation of the robot is ensured.

Description

Technical Field

[0001] The present invention relates to the field of robot technology, and in particular to a four-legged line-patrolling robot based on the fusion of gyroscope and visual data. Background Art

[0002] As an important automation equipment, line patrol robots are widely used in industrial automation, intelligent logistics, agricultural plant protection and other fields. At present, the commonly used positioning methods for line patrol robots mainly include gyroscope heading correction, IMU inertial navigation system and visual solution. However, these methods all have certain shortcomings.

[0003] Although the gyroscope can correct the heading, it cannot effectively guarantee the accuracy of the motion path, causing the actual trajectory to often deviate from the expected trajectory. Although the IMU inertial navigation system can work in harsh environments, it is susceptible to noise and accumulated deviations, and the reliability of posture estimation is not high. The visual solution controls the robot by extracting feature information through the camera, but it is difficult to meet the needs of accurate line patrol in poor lighting or insufficient frame rate.

[0004] In existing patents, for example, the patent with publication number CN202110436902.X proposes a line patrol navigation robot and navigation control method. Although the system can improve the positioning accuracy of the robot to a certain extent, it still faces the problem of inaccurate positioning in complex environments. In addition, the patent with publication number CN114705205A proposes a robot visual line patrol navigation method and a visual line patrol robot, but its line patrol effect is not ideal in an environment with large light changes.

[0005] In view of the problems existing in the above-mentioned prior art, this solution proposes a line-patrolling quadruped robot that integrates vision and IMU sensors, aiming to combine the advantages of both and improve the robot's line-patrolling accuracy and stability in various environments. Summary of the invention

[0006] The present invention aims to provide a line patrol quadruped robot integrating vision and IMU sensors, so as to solve the problems of inaccurate positioning and susceptibility to environmental interference of the line patrol robot in the prior art.

[0007] In order to solve the above technical problems, the present invention provides a technical solution: a four-legged line patrol robot based on fusion of gyroscope and visual data, comprising:

[0008] The trunk bracket has eight motor mounting positions at both ends, each of which is equipped with a drive motor, which is connected to and drives a moving leg through a matching coupling; the middle of the trunk bracket has an equipment mounting position, on which is placed a main control device that controls the overall operation of the robot;

[0009] Walking legs, the walking legs include four groups and are respectively assembled at the corresponding positions of the torso bracket. The four groups of walking legs work together to realize the quadrupedal walking of the robot as a whole. Each group of walking legs consists of two thigh brackets, two couplings, two drive motors, two calf brackets, and a walking foot. One end of each thigh bracket is rotatably connected to the motor shaft of the drive motor at the corresponding position through a coupling. Each thigh bracket is connected to a matching calf bracket through a bearing, and the ends of the two calf brackets are assembled on the same walking foot.

[0010] The main control device is a PCB board equipped with a single-chip microcomputer chip, gyroscope, camera, power supply and auxiliary control circuits. The main control device is connected to eight drive motors through signal lines to realize the control of the robot's four-legged movement.

[0011] Furthermore, the gyroscope and camera are each installed on the torso bracket through a matching bracket. The gyroscope detects the speed and acceleration of the robot to obtain IMU information and upload it. The camera extracts visual feature information in the picture and uploads it. The main control device tightly couples the visual feature information with the IMU information nonlinearly to optimize the motion route and achieve precise positioning of the robot. When there is an error in the robot's movement direction, the robot's heading is corrected by controlling the step length of the robot's left and right moving legs.

[0012] Furthermore, the gyroscope transmits the pitch angle, roll angle, and yaw angle data to the main control device. When a yaw angle occurs, the PID closed-loop control is used to change the step length difference of the left and right moving legs to correct the yaw angle; when a pitch angle and roll angle occur, the PID closed-loop control is used to change the difference in leg length of the front and rear, left and right moving legs to keep the robot body level and make the robot run smoothly.

[0013] Furthermore, the PCB of the main control device includes a main board and a slave board. The eight drive motors are directly controlled by two slave boards. The main board calculates the expected output value of the corresponding drive motor through the gait function, sends the expected motor value to the corresponding slave board through CAN communication, and then obtains the actual value of the drive motor at this time through the electronic regulator. The drive motor can be controlled more accurately through the cascade PID closed-loop control.

[0014] Furthermore, the main control device uses three STM32F4 microcontrollers to control eight drive motors.

[0015] Furthermore, the gyroscope uses a nine-axis gyroscope and can measure speed, angle and angular velocity in three directions. The camera is connected to a Raspberry Pi, and the visual code runs on the Raspberry Pi. After the midline error is obtained, the USB port to TTL module of the Raspberry Pi is used to send it to the serial port of the mainboard. The gait function of the mainboard reads the data and performs calculations.

[0016] Furthermore, the working process of the gyroscope includes: S1. If a yaw angle occurs, change the step length of the left and right moving legs, otherwise enter the pitch angle detection; S2. If a pitch angle occurs, change the height difference between the front and rear moving legs, otherwise enter the roll angle; S3. If a roll angle occurs, change the height difference between the left and right moving legs.

[0017] Furthermore, the workflow of the camera includes: S1, initialization and configuration, introducing OpenCV, standard input and output, string processing header files, and introducing serial communication related header files to communicate with the main control device via serial port; S2, image preprocessing; S2.1, image cropping and compression: cropping and compressing the input image to reduce the amount of calculation and focus on the area of ​​interest; S2.2, color space conversion: converting the image from BGR to HSV and grayscale images for subsequent color recognition and edge detection; S2.3, filtering processing: applying bilateral filtering and Gaussian blur to smooth the image and reduce noise interference; S2.4, edge detection: using the Canny operator for edge detection, and enhancing edge features through morphological operations; S2.5, Hough transform: using the Hough line segment detection algorithm to identify straight line segments in the image, and filtering out the left and right boundary lines according to the angle; S33, feature extraction; S3.1, boundary search: searching for the left and right boundaries of the road from the middle to both sides. If the boundary is found, the actual boundary data is updated; otherwise, the line filling process is performed; S3.2, center line repair: update the center line of the expected motion trajectory according to the line filling data of the left and right boundaries; S3.3, linear interpolation: linear interpolation of the center line of the road is performed to ensure the smoothness of the center line data; S4, special element recognition and processing, select three relatively iconic elements as recognition elements; S4.1, zebra crossing recognition: identify zebra crossings by finding contours and filtering rectangles of a specific size; S4.2, obstacle recognition and obstacle avoidance: identify obstacles and adjust the road boundary according to their positions to achieve obstacle avoidance function; S4.3, left and right turn arrow recognition: identify turn signs and adjust the direction of the robot according to the content of the sign; S4.5, error calculation: calculate the error according to the offset of the center line of the road relative to the center of the image, and limit the error range by weighted average method. Then transmit the error to the control system through the serial port to control the step difference between the left and right sides of the robot to correct the error.

[0018] Furthermore, the coordinated workflow of the gyroscope and the camera includes: S1, the gyroscope detects pitch angle, roll angle, and yaw angle data, and the camera detects centerline error data; S2, the height difference between the front and rear marching legs is changed and the data is imported into the gait function; the height difference between the left and right marching legs is changed and the data is imported into the gait function; the yaw angle data and the centerline error data are comprehensively processed to obtain data feedback, the step length of the left and right marching legs is changed and the data is imported into the gait function; S3, after the gait function is processed, the motion adjustment result is fed back to the marching foot for gait adjustment feedback.

[0019] Compared with the prior art, the advantages of the present invention are as follows: the integration of vision and IMU sensors combines the advantages of vision sensors in providing rich information in normal environments and the stability of IMU in providing posture estimation in harsh environments, effectively improving the line patrol accuracy and stability of the robot in various environments; through PID closed-loop control and cascade PID closed-loop control, the step length and leg length difference of the traveling legs can be accurately controlled, thereby correcting the yaw angle, pitch angle and roll angle to ensure the smooth operation of the robot; three STM32F4 single-chip microcomputers are used to control eight drive motors, which improves the control accuracy and response speed, and further improves the line patrol performance of the robot. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a structural schematic diagram of the present invention.

[0021] Figure 2 This is a schematic diagram of the working process of the gyroscope.

[0022] Figure 3 This is a schematic diagram of the camera's workflow.

[0023] Figure 4 This is a workflow diagram for the use of a gyroscope and a camera.

[0024] Figure 5 This is a control connection diagram for the drive motor.

[0025] As shown in the figure: 1. torso support, 2. marching leg, 201. thigh support, 202. coupling, 203. driving motor, 204. calf support, 205. marching foot, 3. main control device. DETAILED DESCRIPTION

[0026] The present invention is further described in detail below in conjunction with the accompanying drawings.

[0027] Embodiment 1

[0028] like Figure 1 As shown, this embodiment provides a four-legged line patrol robot based on the fusion of gyroscope and visual data, including a trunk support, a traveling leg and a main control device. The trunk support is provided with eight motor mounting positions at both ends, and a driving motor is placed on each motor mounting position. The driving motor is connected through a matching coupling and drives and controls a traveling leg. The traveling legs include four groups and are respectively assembled at the corresponding positions of the trunk support. The four groups of traveling legs work together to realize the four-legged movement of the robot as a whole. The main body of the main control device is a PCB board equipped with a single-chip microcomputer chip, a gyroscope, a camera, a power supply and ancillary control circuits. The main control device is connected to the eight driving motors through signal lines to realize the control of the robot's four-legged movement.

[0029] Furthermore, the gyroscope and the camera are each mounted on the trunk bracket through a matching bracket. The gyroscope detects the speed and acceleration of the robot to obtain IMU information and upload it, and the camera extracts visual feature information from the picture and uploads it. The main control device performs tightly coupled nonlinear optimization on the visual feature information and the IMU information to optimize the motion route and achieve accurate positioning of the robot. When there is an error in the robot's motion direction, the robot's heading is corrected by controlling the step length of the robot's left and right moving legs. The gyroscope transmits the pitch angle, roll angle, and yaw angle data to the main control device. When a yaw angle occurs, PID closed-loop control is used to change the step length difference of the left and right moving legs to correct the yaw angle; when a pitch angle and a roll angle occur, PID closed-loop control is used to change the leg length difference of the front and rear, left and right moving legs to keep the robot body level, thereby ensuring the smooth operation of the robot. The PCB of the main control device includes a main board and a slave board, and the eight drive motors are directly controlled by two slave boards. The main board calculates the expected output value of the corresponding drive motor through the gait function, and sends the expected value of the motor to the corresponding slave board through CAN communication. Then the actual value of the drive motor at this time is obtained through the electric regulator, and the drive motor can be controlled more accurately through the cascade PID closed-loop control. The main control device uses three STM32F4 microcontrollers to control eight drive motors to improve control accuracy and response speed. The gyroscope uses a nine-axis gyroscope and can measure speed, angle and angular velocity in three directions. The camera is connected to a Raspberry Pi, and the visual code runs on the Raspberry Pi. After the midline error is obtained, the Raspberry Pi's USB port to TTL module is used to send it to the serial port of the main board, and the main board's gait function reads the data and performs calculations.

[0030] Specifically, the core components of this solution include:

[0031] (1) Trunk support 1

[0032] The trunk support 1 is the main structure of the robot and is made of high-strength lightweight materials to ensure the robot's load-bearing capacity and motion stability. There are eight motor mounting positions at both ends for installing the drive motor 203. It supports the overall structure of the robot and provides an installation position for the drive motor 203. The motor mounting position is fixedly connected to the drive motor 203 by bolts or other fasteners. There is also an equipment mounting position in the middle of the trunk support 1 for installing the main control device 3, and the connection method also uses bolts or other fasteners.

[0033] (2) Marching leg 2

[0034] The traveling leg 2 includes four groups, each group consists of two thigh brackets 201, two couplings 202, two drive motors 203, two calf brackets 204 and a traveling foot 205. The thigh bracket 201 and the calf bracket 204 are made of high-strength lightweight materials to ensure the robot's movement flexibility and carrying capacity. The robot's four-legged travel is achieved by the drive of the drive motor 203. The mutual cooperation of the thigh bracket 201 and the calf bracket 204 enables the traveling leg 2 to swing flexibly, thereby adapting to different terrains and patrol line requirements. One end of each thigh bracket 201 is rotatably connected to the motor shaft of the drive motor 203 at the corresponding position through a coupling 202. The other end of the thigh bracket 201 is connected to a matching calf bracket 204 through a bearing, and the ends of the two calf brackets 204 are assembled on the same traveling foot 205 to form a complete traveling leg 2 structure.

[0035] (3) Main control device 3

[0036] The main body of the main control device 3 is a PCB board equipped with a single-chip microcomputer chip, a gyroscope, a camera, a power supply and an auxiliary control circuit. As the brain of the robot, the main control device 3 is responsible for receiving and processing sensor data from the gyroscope and the camera, calculating the robot's motion state and position information through an algorithm, and controlling the operation of the drive motor 203 to achieve the robot's quadrupedal movement control. The main control device 3 is connected to eight drive motors 203 through signal lines to achieve the robot's motion control. The gyroscope and the camera are each installed on the trunk bracket 1 through a matching bracket, and are connected to the main control device 3 through a signal line to transmit sensor data.

[0037] Based on the above structure, the working principle of this device is as follows: the gyroscope detects the speed and acceleration of the robot to obtain IMU information, and uploads it to the main control device 3. The camera extracts visual feature information from the picture and uploads it to the main control device 3. The main control device 3 performs tightly coupled nonlinear optimization on the visual feature information and the IMU information to optimize the motion route and achieve precise positioning of the robot. When there is an error in the direction of movement of the robot, the main control device 3 corrects the heading of the robot by controlling the step length of the left and right walking legs 2 of the robot. The gyroscope transmits the pitch angle, roll angle, and yaw angle data to the main control device 3, and the main control device 3 uses PID closed-loop control to change the step length difference and leg length difference of the walking leg 2 to correct the yaw angle, pitch angle, and roll angle to ensure the smooth operation of the robot.

[0038] It can be seen that this device integrates vision and IMU sensors, effectively improving the line patrol accuracy and stability of the robot in various environments. Through PID closed-loop control and cascade PID closed-loop control, the step length and leg length difference of the walking legs can be accurately controlled to ensure the smooth operation of the robot. The trunk support 1 and the walking legs 2 made of high-strength and lightweight materials improve the robot's carrying capacity and movement flexibility. The line patrol quadruped robot that integrates vision and IMU sensors provided in this solution has the advantages of simple structure, comprehensive functions, precise control, etc., and has broad application prospects.

[0039] Embodiment 2

[0040] like Figure 1 As shown, this embodiment provides a four-legged line patrol robot based on the fusion of gyroscope and visual data, the core of which is to achieve stable movement of the robot along the predetermined line through the comprehensive control of the main control device 3. The overall working method is described in detail as follows:

[0041] 1. Initialization and calibration

[0042] After the robot is started, the main control device 3 first initializes the system, including calibration of various sensors such as gyroscopes and cameras to ensure data accuracy. At the same time, the main control device 3 sends instructions to put all the traveling legs 2 in the initial position to prepare for the upcoming line patrol task.

[0043] 2. Data Collection and Processing

[0044] When the robot starts to move, the camera captures the surrounding environment in real time, identifies the characteristics of the predetermined route such as color and shape, and transmits this visual information to the main control device 3. At the same time, the gyroscope continuously monitors the robot's attitude angle pitch angle, roll angle, yaw angle and acceleration information, and provides IMU data to the main control device 3.

[0045] 3. Motion Control and Adjustment

[0046] After receiving the data from the camera and the gyroscope, the main control device 3 fuses the visual features and IMU data through a built-in algorithm such as a tightly coupled nonlinear optimization algorithm to calculate the current position, speed and posture of the robot. Based on the processed data, the main control device 3 formulates the next travel strategy, including the travel direction, step length, and whether the posture needs to be adjusted. The main control device 3 sends control instructions to each drive motor 203 through a signal line to adjust the swing angle, speed and step length of the traveling leg 2 so that the robot moves according to the predetermined strategy. If it is detected that the robot deviates from the predetermined route or the posture is unstable, the main control device 3 will immediately start the PID closed-loop control or the cascade PID closed-loop control, and quickly correct the yaw angle, pitch angle and roll angle by adjusting the step length and leg length difference of the traveling leg 2 to ensure that the robot moves smoothly along the correct route.

[0047] 4. Real-time monitoring and feedback

[0048] During the entire line inspection process, the main control device 3 continuously monitors the robot's motion status and changes in the surrounding environment to ensure timely response to any abnormal situation. If a sensor failure, motor overheating or other potential problems are detected, the main control device 3 will trigger an alarm mechanism and send an alarm message to the operator through a preset communication method such as a wireless signal.

[0049] 5. Mission completion and data recording

[0050] When the robot completes the line patrol task or receives a stop command, the main control device 3 will control the traveling leg 2 to slowly stop and enter the standby state. After the task is completed, the main control device 3 will save key data such as position information, posture data, sensor status, etc. during the line patrol process for subsequent analysis and optimization.

[0051] In summary, the line patrol quadruped robot provided in this embodiment realizes precise control and real-time adjustment of the robot's moving process through comprehensive control of the main control device 3 combined with data fusion processing of vision and IMU sensors, ensuring the robot's stable line patrol capability in complex environments.

[0052] Embodiment 3

[0053] like Figure 2 As shown, this embodiment provides a specific implementation method of the gyroscope application. In the prior art, the commonly used positioning methods of line patrol robots have shortcomings. The gyroscope can correct the heading, but it cannot guarantee the motion path, and the actual trajectory often deviates from the expected trajectory. Although the IMU inertial navigation system can work in harsh environments, it is easily affected by noise and accumulated deviations, and the posture estimation is unreliable. The visual solution controls the robot by extracting feature information through the camera. However, when the light is poor or the frame rate is not enough, it is difficult to meet the needs of accurate line patrol.

[0054] In the face of these problems, this embodiment applies the fusion of vision and IMU sensors to the quadruped robot. The vision sensor provides rich information in a normal environment to make up for the insufficient positioning accuracy of the IMU; the IMU provides the robot with posture estimation in a harsh environment to overcome the shortcoming of the vision sensor's high environmental requirements.

[0055] The following figure shows the working process of the gyroscope: S1. If a yaw angle occurs, change the step length of the left and right traveling legs, otherwise enter the pitch angle detection; S2. If a pitch angle occurs, change the height difference between the front and rear traveling legs, otherwise enter the roll angle; S3. If a roll angle occurs, change the height difference between the left and right traveling legs.

[0056] The gyroscope can give the robot three angles. When a yaw angle occurs, the quadruped robot can correct the yaw angle by changing the step length difference between the left and right sides. This process uses PID for closed-loop control. By adjusting the three key parameters, the robot can run more smoothly. Considering the harsh working environment of the quadruped robot, it may fall when the ground is uneven. We transmit the pitch angle and roll angle of the gyroscope to the control system, and the robot body can be kept level by changing the difference in leg length between the front and back and left and right. Even if the road surface is uneven, the robot can adjust the height of the body in time to prevent it from falling during movement.

[0057] Embodiment 4

[0058] like Figure 3 As shown, this embodiment provides a specific implementation method of the camera application, and the workflow of the camera includes: S1, initialization and configuration, introducing OpenCV, standard input and output, string processing header files, and introducing serial communication related header files to communicate with the main control device via serial port; S2, image preprocessing; S2.1, image cropping and compression: cropping and compressing the input image to reduce the amount of calculation and focus on the area of ​​interest; S2.2, color space conversion: converting the image from BGR to HSV and grayscale images for subsequent color recognition and edge detection; S2.3, filtering processing: applying bilateral filtering and Gaussian blur to smooth the image and reduce noise interference; S2.4, edge detection: using the Canny operator for edge detection, and enhancing edge features through morphological operations; S2.5, Hough transform: using the Hough line segment detection algorithm to identify straight line segments in the image, and filtering out the left and right boundary lines according to the angle; S33, feature extraction; S3.1, boundary search: searching for the left and right boundaries of the road from the middle to both sides. If the boundary is found, the actual boundary data is updated; otherwise, the line filling process is performed; S3.2, center line repair: update the center line of the expected motion trajectory according to the line filling data of the left and right boundaries; S3.3, linear interpolation: linear interpolation of the center line of the road is performed to ensure the smoothness of the center line data; S4, special element recognition and processing, select three relatively iconic elements as recognition elements; S4.1, zebra crossing recognition: identify zebra crossings by finding contours and filtering rectangles of a specific size; S4.2, obstacle recognition and obstacle avoidance: identify obstacles and adjust the road boundary according to their positions to achieve obstacle avoidance function; S4.3, left and right turn arrow recognition: identify turn signs and adjust the direction of the robot according to the content of the sign; S4.5, error calculation: calculate the error according to the offset of the center line of the road relative to the center of the image, and limit the error range by weighted average method. Then transmit the error to the control system through the serial port to control the step difference between the left and right sides of the robot to correct the error.

[0059] Embodiment 5

[0060] like Figure 4 As shown, this embodiment provides a specific implementation method for using a gyroscope in combination with a camera, and the specific process includes: S1, the gyroscope detects pitch angle, roll angle, and yaw angle data, and the camera detects centerline error data; S2, the height difference between the front and rear traveling legs is changed and the data is imported into the gait function; the height difference between the left and right traveling legs is changed and the data is imported into the gait function; the yaw angle data and the centerline error data are comprehensively processed to obtain data feedback, the step length of the left and right traveling legs is changed and the data is imported into the gait function; S3, after the gait function is processed, the motion adjustment result is fed back to the traveling foot for gait adjustment feedback.

[0061] In the control system, the PID function is used to map the yaw angle and the centerline error to the step length difference between the left and right sides. If the difference between the two is not large and does not cause the robot to suddenly turn a lot, the foothold of the four legs will be calculated in the kinematic function. If one of the yaw angle and the centerline error exceeds the limit value, the value will be discarded and then calculated. If both exceed the limit value, since the gyroscope is more reliable in most cases, the average of the yaw angle at this moment and the yaw angle at the previous moment is taken for calculation. If the yaw angle data is zero and the centerline error is not zero, then only the centerline error is used for correction, and the yaw angle data is no longer used until the yaw angle returns to zero again. Through the data fusion processing of the two, the static error caused by the IMU is avoided, and the situation where the visual solution cannot work in extreme environments is avoided.

[0062] In addition, Figure 5 As shown, in the implementation of the robot's gait algorithm, this embodiment uses bionics simulation technology to reasonably plan the phase and period of the four legs of the robot dog, and uses a quasi-sinusoidal trajectory to simulate the gait dynamics, and finally uses a kinematic solution to complete the motor's drive of the four-legged position. The foot-end trajectory planning of the quadruped robot is to first design a fuzzy or precise trajectory curve so that the foot end of the leg of the quadruped robot in the swing can swing according to the preset trajectory. In this product design, we used a quasi-sinusoidal trajectory. The control system uses three stm32F4 microcontrollers to control 8 motors, and the eight motors are directly controlled by two slave boards. The output expected value of the corresponding motor is calculated in the main board through the gait function, and the motor expected value is sent to the corresponding slave board through CAN communication. At the same time, the actual value of the motor at this time is obtained through the electric adjustment, and the motor can be controlled more accurately through the cascade PID closed-loop control. The gyroscope uses a nine-axis gyroscope, which can measure the speed, angle and angular velocity in three directions. The camera is connected to a Raspberry Pi, and the visual code runs on the Raspberry Pi. After the centerline error is obtained, it will be sent to the serial port of the mainboard through the USB to TTL module using the USB port of the Raspberry Pi. The gait function of the mainboard reads the data and performs calculations.

[0063] The above shows and describes the basic principles and main features of the present invention and the advantages of the invention. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The above embodiments and descriptions are only for explaining the principles of the present invention. Without departing from the spirit and scope of the present invention, the present invention may have various changes and improvements, which fall within the scope of the present invention. The scope of protection of the present invention is defined by the attached claims and their equivalents.

Claims

1. A quadruped line patrol robot based on gyroscope and visual data fusion, characterized in that include: A trunk support (1), wherein the trunk support (1) is provided with eight motor mounting positions at both ends, each motor mounting position being provided with a driving motor (203), the driving motor (203) being connected to and driving a moving leg (2) via a matching coupling (202); an equipment mounting position is provided in the middle of the trunk support (1), and a main control device (3) for controlling the overall operation of the robot is placed on the equipment mounting position; Walking legs (2), the walking legs (2) comprising four groups in total and respectively mounted at corresponding positions of the trunk support (1), the four groups of walking legs (2) working in coordination to realize the quadrupedal walking of the robot as a whole, each group of walking legs (2) comprising two thigh supports (201), two couplings (202), two drive motors (203), two calf supports (204), and a walking foot (205), one end of each thigh support (201) being rotationally connected to the motor shaft of the drive motor (203) at the corresponding position via a coupling (202), each thigh support (201) being connected to a matching calf support (204) via a bearing, and the ends of the two calf supports (204) being mounted on the same walking foot (205); A main control device (3), wherein the main body of the main control device (3) is a PCB board equipped with a single-chip microcomputer chip, a gyroscope, a camera, a power supply and an auxiliary control circuit. The main control device (3) is connected to eight drive motors (203) via signal lines to realize the control of the robot's four-legged movement.

2. A quadruped line patrol robot based on fusion of gyroscope and visual data according to claim 1, characterized in that: The gyroscope and camera are each mounted on the trunk support (1) via a matching support. The gyroscope detects the speed and acceleration of the robot to obtain IMU information and uploads it. The camera extracts visual feature information from the image and uploads it. The main control device (3) tightly couples the visual feature information with the IMU information to optimize the movement route and achieve accurate positioning of the robot. When there is an error in the movement direction of the robot, the robot's heading is corrected by controlling the step length of the left and right moving legs (2) of the robot.

3. A quadruped line patrol robot based on fusion of gyroscope and visual data according to claim 2, characterized in that: The gyroscope transmits the data of pitch angle, roll angle and yaw angle to the main control device (3). When a yaw angle occurs, the PID closed-loop control is used to change the step length difference of the left and right moving legs (2) to correct the yaw angle; when a pitch angle and a roll angle occur, the PID closed-loop control is used to change the leg length difference of the front and rear, left and right moving legs (2) to keep the robot body level and enable the robot to run smoothly.

4. The quadruped line patrol robot based on gyroscope and visual data fusion according to claim 1, characterized in that: The PCB of the main control device (3) comprises a main board and a slave board, and the eight drive motors (203) are directly controlled by two slave boards. The main board calculates the expected output value of the corresponding drive motor (203) through a gait function, sends the expected motor value to the corresponding slave board through CAN communication, and then obtains the actual value of the drive motor (203) at this time through an electric regulator. The drive motor (203) can be controlled more accurately through cascade PID closed-loop control.

5. The quadruped line patrol robot based on gyroscope and visual data fusion according to claim 4, characterized in that: The main control device (3) uses three STM32F4 single-chip microcomputers to control eight drive motors (203).

6. A quadruped line patrol robot based on fusion of gyroscope and visual data according to claim 1 or 4, characterized in that: The gyroscope uses a nine-axis gyroscope and can measure speed, angle and angular velocity in three directions. The camera is connected to a Raspberry Pi, and the visual code runs on the Raspberry Pi. After the midline error is obtained, the USB port to TTL module of the Raspberry Pi is used to send it to the serial port of the mainboard. The gait function of the mainboard reads the data and performs calculations.

7. The quadruped line patrol robot based on gyroscope and visual data fusion according to claim 1, characterized in that The working process of the gyroscope includes: S1. If a yaw angle occurs, change the step lengths of the left and right moving legs (2), otherwise enter the pitch angle detection; S2, if a pitch angle occurs, change the height difference between the front and rear traveling legs (2), otherwise enter the roll angle; S3. If a rolling angle occurs, change the height difference between the left and right traveling legs (2).

8. The quadruped line patrol robot based on gyroscope and visual data fusion according to claim 1, characterized in that The workflow of the camera includes: S1, initialization and configuration, import OpenCV, standard input and output, string processing header files, and import serial communication related header files, and perform serial communication with the main control device (3); S2, image preprocessing; S2.1, Image cropping and compression: crop and compress the input image to reduce the amount of computation and focus on the area of ​​interest; S2.2, color space conversion: convert the image from BGR to HSV and grayscale image for subsequent color recognition and edge detection; S2.3, filtering processing: bilateral filtering and Gaussian blur are applied to smooth the image and reduce noise interference; S2.4, edge detection: use the Canny operator to detect edges, and enhance edge features through morphological operations; S2.5, Hough transform: Use the Hough line segment detection algorithm to identify straight line segments in the image and filter out left and right boundary lines based on the angle; S33, feature extraction; S3.1, Boundary search: Search the left and right boundaries of the road from the middle to both sides. If the boundary is found, update the actual boundary data; otherwise, perform line filling processing; S3.2, centerline repair: update the centerline of the expected motion trajectory according to the line-filling data of the left and right boundaries; S3.3, Linear interpolation: Linear interpolation is performed on the road centerline to ensure smoothness of the centerline data; S4, special element identification and processing, select three relatively iconic elements as identification elements; S4.

1. Zebra crossing recognition: Identify zebra crossings by finding contours and filtering rectangles of a certain size; S4.2, obstacle identification and avoidance: Identify obstacles and adjust road boundaries according to their locations to achieve obstacle avoidance function; S4.3, left and right turn arrow recognition: recognize the turn sign and adjust the direction of the robot according to the sign content; S4.5, Error calculation: Calculate the error based on the offset of the road centerline relative to the center of the image, and limit the error range through the weighted average method. Then transmit the error to the control system through the serial port, and control the step length difference between the left and right sides of the robot to correct the error.

9. The quadruped line patrol robot based on gyroscope and visual data fusion according to claim 1, characterized in that The cooperation workflow of the gyroscope and the camera includes: S1, the gyroscope detects the pitch angle, roll angle, and yaw angle data, and the camera detects the centerline error data; S2, changing the height difference between the front and rear legs (2) and importing the data into the gait function; changing the height difference between the left and right legs (2) and importing the data into the gait function; after comprehensive processing of the yaw angle data and the midline error data, obtaining data feedback, changing the step length of the left and right legs (2) and importing the data into the gait function; S3. After the gait function is processed, the movement adjustment result is fed back to the walking foot (205) for gait adjustment feedback.

Images